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ADAPTIVE POISSON DISORDER PROBLEM 

By Erhan Bayraktar,^ Savas Dayanik^ and Ioannis Karatzas'^ 

University of Michigan, Princeton University and Columbia University 

We study the quickest detection problem of a sudden change 
in the arrival rate of a Poisson process from a known value to an 
unknown and unobservable value at an unknown and unobservable 
disorder time. Our objective is to design an alarm time which is 
adapted to the history of the arrival process and detects the disorder 
time as soon as possible. 

In previous solvable versions of the Poisson disorder problem, the 
arrival rate after the disorder has been assumed a known constant. In 
reality, however, we may at most have some prior information about 
the likely values of the new arrival rate before the disorder actually 
happens, and insufficient estimates of the new rate after the disorder 
happens. Consequently, we assume in this paper that the new arrival 
rate after the disorder is a random variable. 

The detection problem is shown to admit a finite-dimensional 
Markovian sufficient statistic, if the new rate has a discrete distribu- 
tion with finitely many atoms. Furthermore, the detection problem 
is cast as a discounted optimal stopping problem with running cost 
for a finite-dimensional piecewise-deterministic Markov process. 

This optimal stopping problem is studied in detail in the special 
case where the new arrival rate has Bernoulli distribution. This is a 
nontrivial optimal stopping problem for a two-dimensional piecewise- 
deterministic Markov process driven by the same point process. Using 
a suitable single-jump operator, we solve it fully, describe the analytic 
properties of the value function and the stopping region, and present 
methods for their numerical calculation. We provide a concrete exam- 
ple where the value function does not satisfy the smooth-fit principle 
on a proper subset of the connected, continuously differentiable op- 
timal stopping boundary, whereas it does on the complement of this 
set. 
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1. Introduction and synopsis. Suppose that arrivals of certain events 
constitute a Poisson process N = {Nt;t > 0} with a known rate fj,> 0. At 
some time 9, the arrival rate suddenly changes from to A. Both the dis- 
order time 9 and the post-disorder arrival rate A of the Poisson process 
are unknown and unobservable quantities. Our problem is to find an alarm 
time r which depends only on the past and the present observations of the 
process N, and detects the disorder time 6 as soon as possible. 

More precisely, we shall assume that 9 and A are random variables on 
some probability space {Q,TC,¥), on which the process N is also defined; 
the variables 9, A are independent of each other and of the process A^. An 
alarm time is a stopping time r of the history of the process A^. We shall 
try to choose such a stopping time so as to minimize the Bayes risk 

(1.1) F{t < 9} + cE{t - 9)+ , 

namely, the sum of the frequency P{r < 9} of the false alarms and the 
expected cost cE(r — 9)~^ of the detection delay. 

We shall assume that the post-disorder arrival rate A has some general 
prior distribution z^(-). Similarly, the disorder time 9 will be assumed to have 
an exponential distribution of the form 

(1.2) P{6' = 0} = 7r and ¥{9 > t\9 > 0} = e'^^ t > 0, 

for some vr G [0,1) and A > 0. The Poisson disorder problem with a known 
post-disorder rate (namely, A equals a known constant with probability 1) 
was studied first by Galchuk and Rozovskii [10] and was solved completely 
by Peskir and Shiryaev [15]. In the meantime, Davis [8] noticed that sev- 
eral forms of Bayes risks, including (1.1), admit similar solutions. He called 
this class of problems standard Poisson disorder problems, and found a par- 
tial solution. Recently, Bayraktar and Dayanik [1] solved the Poisson disor- 
der problem when the detection delay is penalized exponentially. Bayrak- 
tar, Dayanik and Karatzas [3] showed that the exponential detection delay 
penalty in fact leads to another variant of standard Poisson disorder prob- 
lems if the "standards" suggested by Davis are restated under a suitable 
reference probability measure. It was also shown [3] that use of a suitable 
reference probability measure reduces the dimension of the Markovian suf- 
ficient statistic for the detection problem, and the solution of the standard 
Poisson disorder problem was described fully. 

We believe that unknown and unobservable post-disorder arrival rate A 
captures quite well real-life applications of change-point detection theory. 
Before the onset of the new regime, past experience may help us at most 
to fit an a priori distribution i/(-) on the likely values of the new arrival 
rate of N after the disorder. Even after the disorder happens, we may not 
have enough observations to get a reliable statistical estimate of the post- 
disorder rate. Indeed, since a good alarm is expected to sound as soon as the 
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disorder happens, we may have very few observations of N sampled from 
the new regime since the disorder. The quickest detection of an unknown 
and unobservable shift in the drift of a Wiener process has been tackled by 
Beibel [4] and Beibel and Lerche [5]. However, we are unaware of any work 
pertaining to Poisson processes. 

Let us highlight our approach to the problem and our main results. We 
show that the most general such detection problem is equivalent, under a 
reference probability measure, to a discounted optimal stopping problem 
with a running cost for an infinite-dimensional Markovian sufficient statis- 
tic. However, the dimension becomes finite as soon as the prior probability 
distribution z^(-) of the post-disorder arrival rate A charges only a finite 
number of atoms. This class of problems is of considerable interest since, in 
many applications, we have typically an empirical distribution of the post- 
disorder arrival rate, constructed either from finite past data or from expert 
opinions on the most significant likely values. 

We then study in detail the case where the new arrival rate after the dis- 
order is expected either to increase or to decrease by the same amount. The 
detection problem turns in this case into an optimal stopping problem for a 
two-dimensional piecewise- deterministic Markov process, driven by the same 
point process. We solve this optimal stopping problem fully by describing e- 
optimal and optimal stopping times and identifying explicitly the nontrivial 
shape of the optimal continuation region. 

The common approach to an optimal stopping problem for a continuous- 
time Markov process is to reformulate it as a free-boundary problem in 
terms of the infinitesimal generator of the process. The free-boundary prob- 
lems sometimes turn out to be quite hard, even in one dimension; see, for 
example, [1, 10, 15]. Here, the infinitesimal operator gets complicated fur- 
ther, and becomes a singular partial differential-delay operator. Moreover, it 
is a nontrivial task, even in two dimensions, to guess the location, shape and 
smoothness of the free-boundary separating the continuation and stopping 
regions, as well as the behavior of the value function along the boundary. 

Instead, we follow a direct approach and work with integral operators 
rather than differential operators. As in [9] and [11] we use a suitable single- 
jump operator to strip the jumps off the original two-dimensional piecewise- 
deterministic Markov process and turn the original optimal stopping prob- 
lem into a sequence of optimal stopping problems for a deterministic process 
with continuous paths. Using direct arguments, we are able to infer from the 
properties of the single-jump operator the location and shape of the optimal 
continuation region, as well as the smoothness of the switching boundary 
and the value function. 

The single-jump operator also suggests a straightforward numerical method 
for calculating the value function and the optimal continuation region. The 
deterministic process obtained after removing the jumps from the original 
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Markov process has two fundamentally different types of behavior. We tailor 
the naive numerical method to each case, by exploiting the behavior of the 
paths. 

We also raise the question when the value function should be a classical 
solution of the relevant free-boundary problem. For a large range of con- 
figurations of parameters, both the value function and the boundary of the 
continuation region turn out to be continuously differentiable, and one may 
also choose to use finite-difference methods for differential-difference equa- 
tions to solve the problem numerically. For a few other cases, we cannot 
qualify completely the degree of smoothness of the value function. Viscosity 
approaches or some other techniques of nonsmooth analysis are very likely to 
fill the gap, but we do not pursue this direction here. We report one concrete 
example on "partial" failure of the smooth- fit principle: in certain cases, the 
value function is continuously differentiable everywhere on the state space 
except on a proper subset of the connected and continuously differentiable 
optimal stopping boundary. 

This work is divided naturally in two parts. In Sections 2-6, we describe 
the problem, formulate a model, and develop an important approximation. 
In Sections 7-11, we use that approximation to develop the solution and 
study its properties. Long proofs are presented in the Appendix. 

2. Problem description. Let N = {Nt; t>0} be a homogeneous Poisson 
process with some rate > on a fixed probability space (r2,'H,Po), which 
also supports two random variables 9 and A independent of each other and 
of the process A^. We shall denote by i^(-) the distribution of the random 
variable A, assume that 

are well defined and finite, 

and that 

(2.2) Po{6' = 0}=7r and Po{6l > t} = (1 - 7r)e-^*, t > 0, 

hold for some constants A > and vr € [0, 1). 

Let us denote by F = {J-'t}t>o the right-continuous enlargement with Pq- 
null sets of the natural filtration cr{Ns ;0 < s <t) of N . We also define a larger 
filtration G = {Gt}t>o by setting Qt = Tt\l o"{0, A}, t > 0. The G-adapted, 
right-continuous (hence, G-progressively measurable) process 

(2.3) /i(t)^Ml{t<e} + Al|t>e|, t>0, 
induces the (Pq, G) -martingale (see [6], pages 165-168) 

(2.4) Zt^exp|^*log(^^^^dAr,-^V(s)-Ai)'is}, ^ > 0. 
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This martingale defines a new probability measure P on every {i},Qt) by 

. , dF 

(2-5) =Zt, t>0. 

Since P and Pq coincide on Qq = a\0 ^K\^ the random variables and A are 
independent and have the same distributions under both P and Pq. 

Under the new probability measure P the counting process has G- 
progressively measurable intensity given by /i(-) of (2.3), namely Nt — Jq h{s) ds, 
t > 0, is a (P, G)-martingale. In other words, the G-adapted process is a 
Poisson process whose rate changes at time 9 from /i to A. 

In the Poisson disorder problem, only the process A^ is observable, and 
our objective is to detect the disorder time 9 as quickly as possible. More 
precisely, we want to find an F-stopping time r that minimizes the Bayes 
risk 

(2.6) Rr{^)=¥{T<9} + c¥.{T-9)+, 

where c > is a constant, and the expectation E is taken under the probabil- 
ity measure P. Hence, we are interested in an alarm time r which is adapted 
to the history of the process A^, and minimizes the trade-off between the 
frequency of false alarms P{r < 0} and the expected time of delay E(t — 9)~^ 
between the alarm time and the unobservable disorder time. 

In the next section we shall formulate the quickest detection problem as 
a problem of optimal stopping for a suitable Markov process. 

3. Sufficient statistics for the adaptive Poisson disorder problem. Let 

S be the collection of all F-stopping times, and introduce the F-adapted 
processes 

A 



(3.1) 



Tl^ ^ r{9 < t\J^t} and 



,fc)^E[(A-M)'l{e<,}|J-i] 



1 — Uf 



Since A has the same distribution u{-) under P and Pq, each G Nq 

is well defined by (2.1). The process 11 = {Ilt,t > 0} tracks the likelihood 
that a change in the intensity of A^ has already occurred, given past and 
present observations of the process. Each $W = {^>f\t > 0}, k G N, may 
be regarded as a (weighted) odds-ratio process. 

Our first lemma below shows that the minimum Bayes risk can be found 
by solving a discounted optimal stopping problem, with discount rate A and 
running cost function f{x) = x — X/c for the F-adapted process 

$(0). By 

Lemma 3.2, the observation process X and the sufficient statistic {^^^^}k>o 
jump exactly at the same times and evolve deterministically between jumps; 
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therefore, their natural filtrations and the collection of their stopping times 
are the same. 

The calculations are considerably easier when the process ^^^^ has the 
Markov property. Unfortunately, this is not true in general. However, the 
explicit dynamics of ^^'^^ in Lemma 3.2 reveal that the infinite-dimensional 
sequence {^'-'^^l^gNf, of the processes in (3.1) is always a Markovian sufficient 
statistic for the quickest detection problem. The same result also suggests 
sufficient conditions for the existence of a finite- dimensional Markovian suf- 
ficient statistic, a case amenable to concrete analysis. 

Lemma 3.1. The Bayes risk in (2.6) equals 

(3.2) i?^(7r) = l-7r + c(l-7r)Eo 

where the expectation Eq is taken under the (reference) probability measure 
Po. 



^\dt 



The proof is very similar to that of Proposition 2.1 in [3]. Note that every 
Zt in (2.4) can be written as 

Lt 

(3.3) Zt = l{t<0} + j^'i-{t>e} 

in terms of the likelihood ratio process 

(3.4) 

Then the generalized Bayes theorem (see, e.g., [13], Section 7.9) and (3.3) 
imply 

Eo[Ztl^e>t}\^t] _ Fo{e > t\rt} _ (1 - 7r)e-^* 



(3.5) 1 - Ut 



Eo[Zt\J^t] EoiZtlJ't] Eo[Zt\Tt] 



since 6 is independent of the process N under Pq and has the distribution 
(2.2). 

Lemma 3.2. Let m'^^\ A; G No, be defined as in (2.1). Then every 
k E No, in (3.1) satisfies the equation 



(3.6) 



C^) - A(m(^) + cDf )) dt + -^t^^\dNt -lidt), t > 0, 



° l-vr 
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Proof. For every A; G No, let us introduce the function 
(3.7) FW(t,rE)^y"(^^y(?;-/i)V(''-^)V(dt;), tGM+,xGM. 

The generaUzed Bayes theorem, (3.5), and the independence of the random 
variables 9, A and the process N under Pq imply that we have 

(,) _ Eo[{A-fi)''Ztl{e<t}\J't] 
— 



{l-nt)Eo[Zt\Tt] 

(3.8) =^^F'^''\t,Nt) + X f\^'^'-'^F'^''\t-s,Nt-Ns)ds 

1 - vr Jo 

= C/f ) + v}'^ 

for every A; G > and t G , where we have set 
C/f ^^FW(t,iVi) and 

(3.9) 

yWA^ f e^(*-^)F('^)(t-s,iVt-iV,)ds. 
Jo 

Every F^^\-,-), k G Nq, in (3.7) is continuously differentiable, and 

(3.10) ^F^''\t,x) = -F^''+^\t,x), t>0,xGR,k£No. 
The change of variable formula for jump processes gives 

FW(t,iVj) = FW(0,0)+ -^{s,Ns)ds + ——{s,Ns^)dNs 

r r)p{k) 

(3.11) - ^ 

= _ f F'^''+^\s,Ns)ds 



+ ^ [FW(s,iV,)-F('=)(s,iV,_)], 



0<s<t 



where AA''^ = A''^ — A^s„ G {0, 1} for s > 0, and the last equality follows 
from (3.10), 



FW(0,0)=m(^^) and / ^i^{s, N^-) dN^ = ^^is,Ns-)AN, 

Jo ax Q^^^^ ax 
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for every integer k>0. However, F^^\s,Ns) — F^^\s,Ns-) is equal to 

since [{v/ fi)^^" — 1] = {ANg/ fi){v — /i). This equation and (3.11) imply 

(3.12) ^ 

+ f -F^''+^\s,Ns-){dNs- fids), tGM+,A;GNo. 
Jo 

This identity will help us derive the dynamics of U^^^ and V^'^^ in (3.9). Note 
that 

(i(^i^C/f =d{e^'F^''\t,Nt)) = e^'F^''\t,Nt)Xdt + e^'dF^''\t,Nt) 

= A^^C/f ^ dt + —F^''+^\t,Nt^){dNt -fidt). 
vr /i 

Therefore, 

dC/f ^ = AC/f'^ + -f/f {dNt -fidt), i > 0, 

(3.13) ^ 

1 — vr 

The derivation of the dynamics of y^*"'^ is trickier. For every fixed s S [0, t), 

t us define A^l'^ = N,^ 
under Pq. As in (3.12), 



let us define N^f^ = Ns+u — Ns, <u <t — s. This is also a Poisson process 



F('^)(t-s,ivii) = mW+ /■ '-F(^-+i)(n,iv(!^)(divW-/idn). 

Jo 

(s) 

Changing the variable of integration and substituting A*", = Ng^, — Ns into 
this equality gives 

F(^)(t -s,Nt-Ns) = m^'^) + - f f'^^+^\v -s,N^-- Ns){dN^ - fidv). 

/i Js 

Let us plug this identity into V^^^ in (3.9), multiply both sides by e""^* and 
change the order of integration. Then 

^-\ty{k) ^ ^^-xs ^^{k) + 1 ^* ^(^=+1) {v - s, N^- - Ns){dN^ - dv)^ ds 
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e-^'F^^+^^ {v -s,N^-Ns) ds^ {dNy - ^ dv) 





^J' Jo 

(k) [\^-\s^^^l_ f\-^^vj''+^\dN^- fidv). 



m 

Jo "'O 
Differentiating both sides and rearranging terms, we obtain 

) = X{m^''^ + Vl^^)dt + -V}^-^^\dNt -fidt), t > 0, 
' = 0. 

Adding (3.13) and (3.14) as in (3.8) gives the dynamics (3.6) of the process 

□ 

Lemma 3.2 shows that the process ^^"^ does not have the Markov prop- 
erty in general. This is because, as (3.6) shows, ^^^^ depends on then 
depends on and so on ad infinitum. However, a finite-dimensional 
Markovian sufficient statistic emerges if the system of stochastic differential 
equations in (3.6) is closeable, namely, if the process ^^''^ can be expressed in 
terms of the processes some A; G Nq. Our next corollary 

shows that this is true if A takes finitely many distinct values. 

Corollary 3.3. Suppose that z^({Ai, . . . , \k}) = 1 for some positive num- 
bers Ai, . . . , Afc. Consider the polynomial 

k k-l 

p(v)^'[[{v-Xi+fi) = v^ + Y,Civ\ veR, 

i=l i=Q 

for suitable real numbers co,...,Cfc_i. Then {^^'^\^^^\ . . . ,^^''~^^} is a k- 
dimensional sufficient Markov statistic, with ^^'^^ = — X]f=o^ Cj<I>^*^ . 

Proof. Under the hypothesis, the random variable p(A — /i) = (A — 
A*)*^ + Yli=o Cj(A — is equal to zero almost surely. Therefore, (3.1) implies 

<Pl + Ci'Pl = -p = 0, F-a.s., for every t>U. 

i=o 1 - lit 

The process on the left-hand side has right-continuous sample paths, by 
(3.6). Therefore, + Q^f ^ = for ah t G M+ almost surely, that is, 
the process ^^^'^ is a linear combination of the processes . . . , □ 

In the remainder of the paper we shall study the case where the arrival 
rate of the observations after the disorder has a Bernoulli prior distribution. 
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4. Poisson disorder problem with a Bernoulli post-disorder arrival rate. 

We shall assume henceforth /i > 1 and that the random variable A has 
Bernoulli distribution 



(4.1) 



Z.({^-l,/i + l}) = l. 



Namely, the rate of the Poisson process is expected to increase or decrease 
by one unit after the disorder. Corollary 3.3 implies that <I>^^^ = and the 
sufficient statistic $(^)) is a Markov process. According to Lemma 3.2, 

the pair satisfies 



(4.2) 



(0) 



A(l + $1°^) dt + -^[^_}{dNt -^idt), 



(0) 



l-vr' 



(4.3) d^ 



(1) 



A(m + $f ^) dt + -^I'HidNt -fidt), 
1^ 



1 



Ml 



(1) 



TT 



-m, 



where, as in (2.1), we set 

(4.4) m = m(^) = Eo[A - /x] = P{A = ^ + 1} - P{A = - 1}. 
The dynamics of the processes 

$(0) and in (4.2) and (4.3) are inter- 
dependent. However, if we define a new process 

(4.5) * : 



"$(0)- 


A 1 


■$(0) 




$(1) 


V2 


$(0) 





then each of the new processes ^^'^^ and <I>(^) is autonomous: 



(0) 



(0) 



(4.6) 



(1) 



(1 — m)TT 
V2(l-vr)' 

(A-l)$f) + 



(0) . A(l-m) 



V2 



X{l + m) 



dt 



-^?2dN, 



V2 



dt + -'^PdNt, 



(1) _ (1 -hm)7r 



The new coordinates ^^^^ and ^^^^ are in fact the conditional odds-ratio 
processes as in 

m = ^ n^ = f^-ho<t\j't} 
* F{e> t\j^t} 

and 

F{A = ^. + l,^<t|.F,} 
^'">t\J^t} 
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Therefore, both ^^^^ and are nonnegative processes. 

Note that m E [—1,1] in (4.4). The cases m = ±1 degenerate to Pois- 
son disorder problems with known post-disorder rates, and were studied 
by Bayraktar, Dayanik and Karatzas [3]. Therefore, we will assume that 
m € ( — 1,1) in the remainder. 



Remark 4.1. For every (/iq G M and e ffi, let us denote by x{t,(f)o), 
t S M, and y{t, (pi), t €M, the solutions of the differential equations 



(4.7) 



d Af 1 — m) 

—x{t, (Po) = (A + l)x{t, (Po) + ^y^. ^(0. M = ^0, 

|y(t, <Ai) = (A - l)y(t, <Pi) + y(0, <Pi) = <Pi, 



respectively. These solutions are given by 



x{t,(po) 



^(A + r 

r A(l + m 



+ 



A(l -m) 



V2(A + 1) 



(4.8) 



y{t,(pi) = < 



V2{X-l) 
1 + m 



A(l + m) 



V2{X - 1) 



+ 



V2 



-t, 



A/1, 
A = l, 



t e M. 



Both x{-,(po) and y{-,(pi) have the semigroup property, that is, for every 
t G M and s G M 

(4.9) x{t + s,cpo)=x{s,x{t,(po)) and y{t + s,cpi) = y{s,y{t,cpi)). 
Note from (4.6) and (4.7) that 



(4.10) 



«.r=x(t-<T„,$W) and ^'^=y{t-anM3, 



(7n<t< cr„+i,n G 



4.1. An optimal stopping problem for the quickest detection of the Poisson 
disorder In terms of the new sufficient statistics <&^^^ and ^^^^ in (4.5), (4.6), 
the Bayes risk of (2.6), (3.2) can be rewritten as 



„ , N c(l — vr) „ 

fi^(7r) = l-7r+ ^ ^ -Eo 
v2 



^V2]dt 
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Therefore, the minimum Bayes risk U (vr) = infi-g^ Rt-{7t), it G [0, 1), is given 
by 



(4.11) 



^ ' V2 VV2(l-^) V2(l-7r)y' 



TT e [0, 1) 



where m is as in (4.4), the function V{-, •) is the value function of the optimal 
stopping problem 



(4.12) 



inf Eg 



g{4>o,4>i) = </'o + </'i 



-V2, 

c 



and Eq ''^^ is the conditional Po-expectation given that 



(0) 




and <1> 



(1) 





It is clear from (4.12) that it is never optimal to stop before the process 
^ leaves the region 



(4.13) 



^o,</'i) g: 



In the next subsection we shall discuss the pathwise behavior of the process 
this will give insight into the solution of the optimal stopping problem 
in (4.12). 

4.2. The sample-paths of the sufficient- statistic process $ = (cl)('')j <^(^)) 
The process ^^^^ jumps downward and increases between jumps; see (4.6). 
On the other hand, the process ^^^^ jumps upward, and its behavior between 
jumps depends on the sign of 1 — A. If A > 1, then the process <I>^^^ increases 
between jumps. If < A < 1, then reverts to the (positive) "mean level" 

between jumps; it never visits (pa unless it starts there; and in this latter 
case, it stays at (pd until the first jump and never returns to (pd (i.e., (j)d> 
is an entrance boundary for <I>'^'^-'). Finally, (pd and 1 — A 7^ have the same 
signs. 

As for the solution of the optimal stopping problem in (4.12), it is worth 
waiting if the process $ is in the region Co of (4.13), or is likely to return 
to Co shortly. The sample-paths of the process $ are deterministic between 
jumps, and tend toward, or away from, the region Co- These two cases are 
described separately below. In both cases, however, the process $ jumps 




in the same direction relative to its position before the jump. A jump at 
{(f)Q,4>i) is an instantaneous displacement <^o 'Pi]'^ ii^ ^- Therefore, 

the jump direction is away from (resp. toward) the region Cq if (po < 
(resp. (l)o> (pi). Along a quarter of a circle in Figure 1(a), the directions of 
jumps at an equal distance from the origin are illustrated by the arrows. 
Note also that, along any fixed half-ray in M^, the jump direction [namely, 
the angle a in Figure 1(a)] does not change, but the size of the jump does. 

4.3. Case I: A ^Harge" disorder arrival rate. Suppose that A > 1 or < 
{X/c)V2 < (pd. Equivalently, A > [1 - (1 + m)(c/2)]+ is "large." Between 
jumps, the process $ gets farther away from the region Co- It may return 
to Co by jumps only, and only if the jump originates in the region L = 
{{<po,(pi) -.(pQ > (pi}; see Figure 1(a). But, if ^^^^ reaches at or above (A/c)\/2, 
then $ will never return to Cq. 

4.4. Case II: A ^^small" disorder arrival rate. Now suppose that < 
(pd < iX/c)V2. Equivalently, 0<A<1-(1+ m)(c/2) is "small." If the pro- 
cess $ finds itself in a very close neighborhood of the upper-left corner of 
the triangular region Cq, then it will drift into Cq before the next jump 
with positive probability. Otherwise, the behavior of the sample-paths of $ 
relative to Cq is very similar to that in Case I; see Figure 1(b). 

5. A family of related optimal stopping problems. Let us introduce for 
every n G N, the optimal stopping problem 



K(.^o,<Al)=mf]E^*^ 



res 



r°"e-^(il»U[")<i* 
Jo 



14 



E. BAYRAKTAR, S. DAYANIK AND I. KARATZAS 



(5.1) 

(./>0,</.i)Gffi^, 

obtained from (4.12) by stopping the process $ at the nth jump time cr„ of 
the process A^. Since g{-, ■) in (4.12) is bounded from below by the constant 
— (A/c)\/2, the expectation in (5.1) is well defined for every stopping time 
r G 5. In fact, —\/^/c <Vn<0 for every n G N. Since the sequence (cr„)„>i 
of jump times of the process is increasing almost surely, the sequence 
(Vn)n>i is decreasing. Therefore, lim„^oo exists everywhere. It is also 
obvious that Vn>V, n G N. 



Proposition 5.1. As n — > oo, the sequence ^n('Ao)'Ai) converges to 
V{(po,(pi) uniformly in {(j)Q,(pi) G M^. In fact, for every n G N and {(po,(pi) G 
Ml, we have 



(5.2) 



V2 



>K(</>o,'/>i)-^(</'o,</'i)>0. 



Proof. Fix ((^o, (Ai) G M^. For r G 5, n G N, we express E^''''^^ [Jq e"-^" x 
g{^s)ds] as 



>Vn{<p0,<Pl) 



e-^'g{^s)ds 

tA(T„, 



V2 / Y 



l{r>a„} / e~^'g{^s)ds 

J<Jn 



e-^'g{^,)ds 



A 



^/2 • E^"''^^ 



Hr>a„] / e ^'ds 



We have used the bound g{4>o,(f>i) > — (A/c)\/2 from (4.12), as well as the 
fact that A?^ is a Poisson process with rate fi under Pq, and (T„ is the nth 
jump time of N. Taking the infimum over r G 5 gives the first inequality in 
(5.2). □ 



We shall try to calculate now the functions Vn{-) of (5.1), following a 
method of Gugerli [11] and Davis [9]. Let us start by defining on the collec- 
tion of bounded Borel functions w : Ml R the operators 



Jw(t,(j)o,(j)i) 



:E7'' 



(5.3) 



thai 



+ l|,>.,}e"^'^^«;(cI>W,l>«) 



(5.4) Jtw{(j)Q,(f)i) = inf Jw{u,(j)Q,(j)i) for every t G [0, oo]. 

uS[t,oo] 
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The special structure of the stopping times of jump processes (see Lemma A.l 
below) implies 

r-rAfJi 



(5.5) 



Jow{(j)o,(pi) = inf Eg 



'0,<pi 



Jo 

+ l{,>.,|e-^<^^u;($(°),$(V) 



By relying on the strong Markov property of the process N at its first jump 
time (Ti, one expects that the value function V of (4.12) satisfies the equation 
V = JqV. Below, we show that this is indeed the case. In fact, if we define 
Vn : 1-^ M, n E No sequentially by 

(5.6) vo = and Vn — JoVn-i VnGN, 

then every Vn is bounded and identical to Vn of (5.1), whereas lim„^oo^^n 
exists and is equal to the value function V in (4.12). 

Under ¥o, the first jump time ai of the process has exponential distri- 
bution with rate Using the Fubini theorem and (4.10), we can write (5.3) 
as 

(5.7) Jw{t,(^o,(^i)= f\~^^+''^^{g + fi-wo S)ix{u,(t)o),yiu,cPi))du 



for every t£ [0,oo], where x{-,(j)Q) and y(-,(/>i) are the solutions (4.8) of 
the ordinary differential equations in (4.7), and 5:]^^ is the linear 

mapping 

(5.8) sicl)o,<Pi)=((l--)^o,(l+^ 



Remark 5.2. Using fi> 1 and the explicit forms of x{u, ^o) and y{u, (pi) 
in (4.8), it is easy to check that the integrand in (5.7) is absolutely integrable 
on M_|_. Therefore, 

lim Jw{t, (po^fpi) = J'w{oo, (/>o, 0i) < oo, 

t—tOC 

and the mapping 1 1— > Jw{t,(f>Q,(f>i) : [0,oo] i— > M is continuous. The infimum 
Jtw{4>o,(pi) in (5.4) is attained for every t £ [0,oo]. 

Lemma 5.3. For every bounded Borel function w : M, the mapping 

Jqw is bounded. If we define \\w\\ = sup(^^^^^^^^^2 \w{(t)o,(j)i)\ < oo, then 



X V2 u 

+ 



(5.9) vA + /x c X + fi 



\w\\^ < Jow{(t)o,(t>i) < 0, 



bo,cl)i)GRl. 



If the function w{(j)Q,(f)i) is concave, then so is Jow(</>o, <Ai)- If wi<W2 are 
real-valued and bounded Borel functions defined on R^, then JqWi < JqW2- 
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Corollary 5.4. Every Vn, n G No, in (5.6) is bounded and concave, 



exists, and is also bounded and concave. 

Both Vn : 1-^ M, n G N, and v : i-^ M are continuous, increasing in 
each of their arguments, and their left and right partial derivatives are 
bounded on every compact subset o/M^. 



Proposition 5.5. For every n G N, the functions v^ of (5.6) and Vn of 
(5.1) coincide. For every e > 0, let for every n = 0, 1, . . . , (cpOjCpi) G M^, 

rni4>o,4'i) - inf{s G (0, oo] : J7;„(s, (/-o, (/>i) < JoVn{4>o,4'i) + e}, 
Si = rg($o) A cJi and 



Proposition 5.6. We have v{(j)o,(j)i) = V{(po,(l)i) for every (0o,</'i) G 
. Moreover, V is the largest nonpositive solution U of the equation U = 
JoU. 

Lemma 5.7. Let w.M.'^ be a bounded function. For every t G 
and {(po,(pi) G R\, 

(5.12) Jtw{<))o, 0i) = Jw{t, Ai) + e^(^+^)* Jou;(x(t, c^o), y(t, ^i))- 

Corollary 5.8. Let 

(5.13) r„(0o,</'i) =inf{s G (0, cx)] : J-t;„(s, (0o,<Ai)) = -^o^^n(</'o, 0i)} 
be the same as r^((/)o, (/>i) in Proposition 5.5 wzi/i e = 0. Then 

(5.14) r„((/)o,0i) = mf{t>O:?;„+i(x(t,(/.o),y(t,0i)) = O} (inf0 = oo). 

Proof. Let us fix ((/)o,(/>i) G M^, and denote r„((/>o, </>!) by r„. By Re- 
mark 5.2, we have JVn{rn,(pO,<Pl) = JoVn{(l)0,4>l) = Jr„Vn{4>0,4>l)- 

Suppose first that r.„ < oo. Since JqVu = Vn+i, taking t = r^ and w = Vn 
in (5.12) imphes that Jv„(r„, ^o, (^i) equals 

«^r„'yn(0O,0l) = JWn('^n,(Ao,<Al) + 6"^^+^^'"" (2;(r„ , (/)o) , y (r„ , (/)i)) . 




■+' 




where 9s is the shift- operator on Q: Nt o Os = Ng^f Then 
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Therefore, f„+i(a;(r„, (/)o), y(r„, = 0. 

If < t < r„, then Jt'„(t, (?!)o, 0i) > JoVn{(l)o,4>i) = Jr„Vn{4'o,4>i) = 
JtVni4'Oj4>i) since JuVn{4>Q-,4>i) is nondecreasmg. Taking t G (0,r.„) and 
w = Vn in (5.12) imphes 

JoVn{4>0, 4>l) = JtVni<t>0,(t>l) 

= Jvn{t, (Ao, 0i) + e-'-^+f'^'vn+iixit, (t>o),y{t, (Al)). 

Therefore, Vn+i{x{t,(j)Q),y{t^(l)i)) < for every t G (0,r„), and (5.14) fohows. 

Suppose now that r„ = oo. Then we have Vn+i{x(t,4>o),y{t,(pi)) < for 
every t G (0, oo) by the same argument in the last paragraph above. Hence, 
{t > 0:Vn+i{x{t,(f)o),y{t,(l)i)) = 0} = 0, and (5.14) stih holds. □ 

Remark 5.9. For every t G [0,rn{(l)Q,(pi)], we have JtVn{(t>Q-,4>i) = 
JoVn{(t>o,4>i) =Vn+i{(t>Oi(t>i)- Then substituting ?i;(-, •) =Vn{-,-) in (5.12) gives 
the dynamic programming equation for the family {fA:(-, OlfeeNo • for every 
(00, ^i) effi+ and nGNo, 

,^ ^'n+l(</'O,0l) = JVnitAo^ft^l) + er''^^^^*Vn+l{x{t,(t)o),y{t,(j)i)), 

(5.15) 

t G [O,r„(0o,0i)]. 

Remark 5.10 (Dynamic programming equation). Since y(-, •) is bounded, 
and V = JqV by Proposition 5.6, Lemma 5.7 gives 

^^•^^^ iGM+, 
for every [cjiQ, 4>i) & M^; and if we define 

r{(t>o,(t^i) = inf{t > : JF(t, 00, (/.i) = JoF(0o, 0i)}, 

(5.17) 

(0O,0i)GM2+, 

then arguments similar to those in the proof of Corollary 5.8, and (5.16), 
give 

(5.18) r(0o, 0i) = inf{i > : V{x{t, cPo),y{t, 0i)) = 0}, (0o, 0i) G M^, 
as well as the dynamic programming equation 

y(0o, 0i) = JV{t, 00, 0i) + e-(^+'^)V(x(t, 0o), y(t, 0i)), 

(5.19) 

tG [O,r(0o,0i)], 

for the function V{-, •) of (4.12). Because 1 1— > Jwit, (0o, 0i)) and t ^ Jtw{(j)o, 
01 ) are continuous for every bounded i(;:M^ h^M [see, e.g., (5.7)], the iden- 
tity (5.16) implies that t>-^ V{x{t,(po),y{t,4>i)) is continuous. Therefore, ev- 
ery realization of t ^ V{^t) is right-continuous and has left limits. 
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Let us define the F-stopping times 

(5.20) Ue = mi{t>0:V{^t)>-e}, e>0. 
By Remark 5.10, we have 

(5.21) V{^u,) > -e on the event {Ue < oo}. 

Proposition 5.11. Let Mt = e'^^Vi^t) + /q e-^^c/(*^)ds, t > 0. For 
every n G N, e > and (c^o, ^i) ^Rl, we have E^"'"^^ [Mq] = E"^"'^' [Mu.AaJ, 
that is, 



(5.22) V{cPo,(l^i)=Et°''^' 



Jo 



Proposition 5.12. For every e>0, the stopping time Us in (5.20) is 
e- optimal for the problem (4.12), that is, 



<V{(j)o,(pi) +e for every {<j)o,(pi) £ . 



f \-^'g{^s)ds 
Jo 



6. A bound on the alarm time. We shall show that the optimal contin- 
uation region C = {(i^Oi 4'i) ^ ^+ '■ V{4'o,4'i) < 0} is contained in some set 



i? = {(<^O,</'l)GlR+:0O + 0l<n 

(6.1) 

for a suitable ^* G 



— -^\/2, oo 



Therefore, the region C has compact closure; this will be very useful in prov- 
ing in the next section that C has a boundary which is strictly decreasing 
and convex. 

Recall from Section 4.1 that it is not optimal to stop before the process 
$ leaves the region Co in (4.13). Thus, the optimal stopping time Uq of 
Proposition 5.12 is bounded from below and above as in 

Tco = inf |t > : $f + ^^'> > <Uo<TD 

(6-2) 

^inf{t>0:$J°^+$f^ >r} 

in terms of the exit times tcq and td of the process $ from the regions 
Co and D, respectively. The constant threshold ^* in (6.1) is essentially 
determined by the number (A -|- /i)\/2/c [see (6.5), (6.9) and (6.11)], and 
our calculations below suggest that they are close. Therefore, the bounds in 
(6.2) may prove useful in practice. The difference [(A + /x)/c]\/2 — (A/c)\/2 = 
{fi/c)\/^ between the thresholds that determine the latest and the earliest 
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alarm times is also meaningful. It increases as ///c increases: waiting longer 
is encouraged if the new information arrives at a rate higher than the cost 
for detection delay per unit time when the disorder has already happened. 

Finally, we prove in Lemma 6.1 that td in (6.2) has finite expectation. 
Therefore, 

Et°'^' [Uo] < E^0'<^^ [td] < oo for every (00, (/>i) G M^. 

Let T £ S he any F-stopping time. By Lemma A.l, there is a constant 
t > such that t A ai = t A ai almost surely. Therefore 



<t>o,4'i 



(6.3) 



e-^'g{^s)ds 







l{s<ai}e ^''g{x{s,cl)o),y{s,cl)i))ds 



^ Jff<t>0,'t>l 



[^{t>ai}t 



-Ao-i] 



-(A+/i)s 



g{x{s,(f)o),y{s,(l)i)) - -V2 



ds. 



The inequality follows from gi(po,(j)i) > ^(0,0) = -{X/c)V2; see (4.12). The 
functions x(-,(/)o) and y{-,(pi) are the solutions of (4.7) (see Remark 4.1), 
and cJi has exponential distribution with rate /U under Pq. Clearly, if for 
< s < cxo we have 



(6.4) < g{x{s, 0o), y(s, (pi)) --V2 = x{s, 4>o) + y{s, (/)i) 



V2, 



then (6.3) implies that Eq'^''^^ [Jq e~''^'^g{^s) ds] > for every F-stopping time 
r 7^ almost surely (since the filtration F is right-continuous, the probability 
of {t > 0} G J-Q equals zero or 1). Thus, stopping immediately^^ is optimal 
at every ((^Oi<^i) for which (6.4) holds. 

If A > 1, then s i-^ x{s,4>o) and s y{s,(f)i) are increasing for every 
{(pOjCpi) (^M.'^; see (4.7) and Figure 2(a). Therefore, x{s,cl)o) + y{s,(t>i) > 
x{0,(I)q) + y{0,<pi) = + (pi for every < s < oo. Hence, (6.4) holds, and 
therefore it is optimal to stop immediately outside the region 



(6.5) 



^)0,(/>i) GMl :0o + 0l < 



■V2 



if A> 1. 



Suppose now that < A < 1; equivalently, (pd of (4.14) is positive. Then 
s ^ x{s,(po) is increasing for every cpQ G M+. For (pi = (p^, the derivative 
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dy{s, (j)d)/ds in (4.7) vanishes for every < s < oo. The mapping s y(s, (j)i) 
is increasing if € [0, decreasing if (f)i G ((/)f;,oo), and (f)[s^4)d) = 4>d for 
every < s < cxo; see (4.7) and Figures 2(b),(c). The derivative 

(6.6) ^[x{t, <Po) + yit, (/-i)] = (A + l)x{t, <Po) + (A - l)yit, 0i) + XV2 
at 

of the right-hand side of (6.4) [see also (4.7)] vanishes if s i— > {x{s, (j)o),y{s, (pi)) 
meets at s = t the hne 

(6.7) ^:(A + l)x + (A- l)y + AV2 = or y = \^x + -^V2. 

1 — A i — A 

Since m G (—1,1), the "mean-level" (pd in (4.14) and the y-intercept of the 
line £ in (6.7) are related as in 



4>d ■ 



A 1 + m A /- 
< z rV2. 



1- A ^ 1-A 



Because I is increasing, this relationship implies that the line £ in con- 
tained in M+ X {(j)d, oo) [see Figure 2(b), (c)]. However, every curve t i— > 
(x(t, (/)o), y(t, 0i)) starting at some {(f)o,(l)i) in M+ x {(f)d,oo) is "decreas- 
ing," and the derivative in (6.6) is increasing. Therefore, any curve t 
{x(t, (po) , y{t, (pi)) , {(pOj'Pi) £ niay meet £ at most once, and 



(6.^ 



if 1 1— > {x{t,(j)o),y{t,4>i)) meets the line £ at ti = L£{(pQ,(pi), 
then t x(t,(l)o) + y(t,(j)i) is decreasing (resp., increasing) 
on [0, ti] (resp., on [t£,oo)). Otherwise, 1 1— > x{t, cpo) + y{t, cpi) 



is increasing on [0, oo). 

Consider now the first of two possible cases: the line i does not meet Di 
of (6.5); that is, A/(l - A) > (A + ^)/c, as in Figure 2(b). Then (po + 4>i > 




Fig. 2. 
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(A + /x)V2/c for every (0o,0i) G i?. Therefore, (6.8) implies that (6.4) holds, 
that is, it is optimal to stop immediately, outside 

(6.9) D, = |(0o,0i)GM^:<^o + '^i<^V2} if > 

In the second case, the line I of (6.7) meets the region Di, that is, < 
A/(l — A) < (A + /u)/c; see Figure 2(c). Let us denote by the point 

at the intersection of the line i and the boundary x + y — (A + \x)\pllc = 
of the region Di. By running the time "backward," we can find ^* (and t*) 
such that 

(6.10) {o,n = {^i-t*,<t>*o),y{-t*,<Pl)). 

Indeed, using (4.8), we can obtain first t* > by solving = rc(— t*, (/)q), and 
then ^* =y{—t*,(f)\). By the semigroup property (4.9), we have 

x(r , 0) = x{t\x{-t\ (pD) = x{t* + (-r ), <AS) = x(o, <aS) = 4>o, 

yit*,e) = y{t*,yi-t\cPl)) = y{t* + i-t*),cPl) = y(0, cPl) = cPl. 

Hence, the curve {x{t,0),y{t,^*)), t>0, meets i at ((/)q,0J), and t£ in 
(6.8) equals t*; see Figure 2(c). Therefore, (6.8) implies that 

x{t, 0) + y{t, C) > x{t*,0) + yit*,C) = ^o + ^l = < t < oo. 

In particular, ^* = + ^ = 2;(0, 0) + y{0,C) > (A + /i)\/2/c. We are now 
ready to show that it is optimal to stop immediately outside the region 

(6.11) D2^m,4>i)eM.l:cj)o + (l>i<n ifO<-A^<^, 

1 — A c 

where ^* is as in (6.10). The curve th^ (x(t, 0), y(t, ^*)) divides M.\ into two 
connected components each containing the region Di of (6.5) and 

M^{M.l\ D2) n {(x, y) G : (A + l)x + (A - l)y + \V2 < 0}, 

respectively [see (6.7)]. Every curve 1 1— > {x{t,(pQ),y{t,(pi)), t>0, starting at 
(^O)i^i) ill M will stay in the same component as M. Therefore, the curve 
intersects the line i away from Di, and (6.8) implies that (6.4) is satisfied 
for every {(pQ,(pi) £ M. 

For {ct>o,(t>i) G (M^\L'2)n{(x,y) GM^:(A + l)x + (A-l)y + AV2>0},the 
curve 1 1— > (x(i, c/iq), y(t, (pi)), t>0, does not meet i; therefore, 1 1— > x{t, (po) + 
y{t,4)i) increases by (6.8) and 

x{t, (j)o) + y{t, 0i) > a;(0, cpo) + y(0, (pi) 

= (po + (pi>C>^^^V2, 0<s<oo. 
c 

Thus, the sufficient condition (6.4) for the optimality of immediate stopping 
holds for every ((/>o, (/>i) G \ Z)2- 
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Lemma 6.1. Let to he the exit time of the process $ from the region D 
in (6.1). Then Eq"'"^"^ [r/j] is finite for every [(jiQ^cjii) G M^. 

Proof. Let /(0o,(/>i) = 4>q + 4>i, {(j)o,4>i) £ M^. Using the explicit form 
of the infinitesimal generator A of the process $ in (A. 4), we obtain 

Af{<i>o, <Pi) = (A + m + + (A - m + + 



V2 ' V2 

(6.12) +/1 ^ 

= A((/.o + 0i + ^) > aV2 



for every ((/>o,(/>i) G M^. Since /(•,•) is bomided on D of (6.1) and td At, 
t > 0, is a bounded F-stopping time, (A. 3) holds for t = td At. Then we 
have 

e{l + -^>^t'^^[f{^ruM)] 

(6.13) =/((^o,<Ai)+IEr'^^ 



<PO,<Pl 



> AV2E;J«''^^[rz?At], t>0. 



The process $ may leave the region D in (6.1) continuously or by a jump. 
Since /(5(0o, </>!)) = (1 + l//^)</'o + (1 - l//^)0i < (1 + l/^)(0o + 4>i) = (1 + 
1/ fi)f{(j)o,(pi) < (1 + ^/fJ-)^* for every ((^Oif/*!) G -D, and this upper bound is 
larger than the first inequality in (6.13) follows. The second inequality is 
due to (6.12). Finally, the monotone convergence theorem and (6.13) imply 
that E;J°''^i[tz)] is finite. □ 

7. The solution. In Proposition 5.1, we showed that the function V{(j)o,(j)i) 
of our original optimal stopping problem in (4.12) is approximated uniformly 
in {(j)Q,(pi) £ by the decreasing sequence {V^(<^0; i;^i)}neN of the value 
functions of the optimal stopping problems in (5.1). The value functions 
Vn{-, •) = Vn{-, •), n G N can be calculated sequentially by setting vq = 0, and 

Vn+l{4>0,4>l) = JoVn{(f>0,4>l) = mf J Vn{t, , 
,^ -j^N tG[0,oo] 

where the operator J is defined in (5.3); see Proposition 5.5. 

Finding the infimum in (7.1) is not as formidable as it may appear. By 
Proposition 5.5, the infimum in (7.1) is always attained [i.e., the case e = 
in (5.11)]. By Corollary 5.8, it is attained at the time, r.„((/>o, (/>i), the 
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deterministic continuous curve ti— > (x{t,(po),y{t,(l)i)) in (4.7) exits from the 
set 

{(</.o, (Ai) G M.I : Vn+i{ct>o, <Ai) < 0} C {(00, (t>i) e K ■ ^('^0, <Al) < 0} C Z?; 

here is the triangular region in (6.1), and the last inclusion is proven in 
Section 6. Therefore, the search for the infimum in (7.1) can be confined for 
every n S N to 

i-t 

(7.2) 

tG[O,f(0o,0i)], 

over the interval t £ [0,f((/)o,(/>i)], where 

ficj)o,cj)i) ^ mi{t > : x{t, (/.q) + y{t, 4>i) > T}, ('Ao, </>!) G K^, 

is the (bounded) exit time of the curve 1 1-^ {x{t, (pQ),y{t, 0i)) from the region 
D of (6.1). 

Finally, the error in approximating V{-,-) of (4.12) by {i'n(T)}neN in 
(7.1) can be controlled. For every e > 0, 



V2/ fi 

c \X + fi, 



(7.3) 

(</'0,</'i)GK^, 

by Propositions 5.1 and 5.5. The exponential rate of the uniform convergence 
of {i'n('; OInGN to V{-,-) on in (7.3) may also reduce the computational 
burden by allowing relatively small number of iterations in (7.1). 

In the remainder, we draw attention to certain special cases where the 
value function V{-,-) can be calculated gradually at each iteration in (7.1); 
see Proposition 8.3. In the meantime, we shall give a precise geometric de- 
scription of the stopping regions 

(7.4) Tn = {{(Po,(Pi)€Rl:Vn{<PoAi) = 0}, C„^M^\r„, n G N, 

(7.5) r^{(0o,</'i)GM^:7;(</.o,(/)i)=O}, C ^ \ T, 
and describe the optimal stopping strategies. 

8. The structure of the stopping regions. By Proposition 5.12, the set T 
is the optimal stopping region for the problem (4.12). Namely, stopping at 
the first hitting time Uq = inf{t G M+ : *t G T} of the process * = ($(0), $(!)) 
to the set T is optimal for (4.12). 

Similarly, we shall call each set r„, n G N, a stopping region for the family 
of optimal stopping problems in (5.1). However, unlike the case above, we 
need the first n stopping regions, Fi, . . . , r„, in order to describe an optimal 



24 



E. BAYRAKTAR, S. DAYANIK AND I. KARATZAS 



stopping time for the problem of (5.1). Using Corollary 5.8, the optimal 
stopping time 5„ = in Proposition 5.5 for Vn of (5.1) may be described 
as follows: Stop if the process $ hits r„ before N jumps. If N jumps before 
$ reaches r„, then wait, and stop if ^ hits r„_i before the next jump of N, 
and so on. If the rule is not met before the (n — l)st jump of N, then stop 
at the earliest of the hitting time of Ti and the next jump time of N . See 
Figure 3(b) for three realizations of the stopping time 5*2. 

We shall call each — \ Tni n E N, a continuation region for the 
family of optimal stopping problems in (5.1), and C = \ T the optimal 
continuation region for (4.12). The stopping regions are related by 

Rl\DcTc---CTnC r„„i C • • • C Ti C Mi \ Co and 

OO 

n=l 

since the sequence of nonpositive functions {unjneN is decreasing, and v = 
lim^^oo i Vn by Lemma 5.4. The sets D and Co are defined in (6.1) and 
(4.13), respectively. Since n S N and v are concave and continuous map- 
pings from into (—00,0] by Lemma 5.4, the stopping regions r„, n G N, 
and r are convex and closed. Let us define the functions 7„:M+ i-^ M+, 
n E N, and 7 : M+ by [see also Figure 3(a)] 

7„(x)^inf{yGM+:(x,y)Gr„}, xeR+, 

j{x)=mf{yeR+:{x,y)eT}, x G M+, 




Fig. 3. (a) The stopping regions (eaeh arrow at the boundary of a region points toward 
the interior of that region), and (b) three sample-paths and the optimal stopping times S2 
and Uo for the optimal stopping problems V2 in (5.1) and V in (4.12), respectively. 
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and the numbers 

^„=inf{xGM+:7„(x) =0}, n E N, and ^ = inf{a; G M+ : 7(2;) = 0}. 

Then the stopping regions Tri, tt. G and T are the convex and closed 
epigraphs of the functions 7n(-)i nG'N, and 7(-), respectively. Therefore, 
7n(-)) n G'N, and 7(-) are convex and continuous mappings from M_|_ into 
M+. 

By the set-inclusions in (8.1), we have (A/c)\/2 < £,n~i < ■^n <i C < ^^v 
the same ^* G IR+ in the description (6.1) of the set D. Since Vn, n G N, and 
V vanish on M+ \ L» = {((/>o, </>i) G : (^0 + (/"i > C*} by (8.1), the functions 
7n(-)i S N, and 7(-) vanish on [^*,oo). However, and ^ are the smallest 
zeros of the continuous functions 7n(-)) ^t- G N, and 7(-), respectively. Since 
both functions are also nonnegative and convex, the function 'jni'), f^ G N 
[resp. 7(-)] is zero on [^„,oo) (resp. on [^,00)) and strictly decreasing on 
[0,^„] (resp. on [0,^]). For future reference, we now summarize our results. 

Proposition 8.1. There are decreasing, convex and continuous map- 
pings 'jn '■ IR+ n£N, and 7 : R+ 1-^ IR+ such that 

r„ = {(0o,'Ai)elRl:<^i>7n(0o)}, nGN, 

r = {((/>o,<Ai)eIRl:<^i>7(</'o)}. 

The sequence {7n(0o)}neN is increasing and j{(j)o) = lim ] 7n(0o) for every 
(pQ G M_|_ . There are numbers 

(8.2) -V2 < 6 < • • • < ^n-i < e« < • • • < e < r < 00 

c 

such that 7n(-)) nGN [resp., is strictly decreasing on [0, nGN 

{resp., [0,^,]) and vanishes on [,^„,oo), nGN {resp., [^,,00)). Moreover, 

(8.3) -V2 < 71(0) < • • • < 7n-i(0) < 7n(0) < • • • < 7(0) < r < 00. 
c 

The number ^* is the same as in the definition of the set D in (6.1). 

Notation 8.2. Let S-.R"^ h^M^ be the same linear map as in (5.8). 
(Nl) For any subset -R C m2_, 

5-("+i)(i?) = 5-i(5-"(i?)), nGN, 
S-\R)^{{x,y)eRl:S{x,y)eR}, 
5"+i(i?)4s(S"(i?)), nGN, 
S{R) ^ {S{x,y) eRl:{x,y) € R}, 
and S^{R) = S{S-^{R)) = S-^S{R)) = R. 
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(N2) For every singleton {{x,y)} C M^, we write 

S"'{{x,y}) = S^{x,y) = (^(^1 - (^1 + , m G Z. 

(N3) For any function g : ]R+ , we define the function [51] : 1-^ , 

n G Z by 

^ inf{y G M+ : (x,y) G 5"(epi(5))}, x G M+. 

That is, 5" [5] is the function whose epigraph is the set S^{epi{g)). Note 
that we use and to distinguish the sets and the functions. 

(N4) For every subset R of M^, we denote by cl(i?) its closure in and by 
int(-R) its interior. We shall denote the support of a function (7:]^+ ^ 
R+ by 

supp(5) = cl({xGM+:g(x) >0}). 

The process $ jumps into the region T [resp. /^""(r), n G N] if the process 
N jumps while $ is in the region S-^T) [resp. S-("+i)(r), n G N]. Clearly, 
if the process $ can never leave the region ^"^(r) before a jump, then 
the value functions V{-,-) and Vi(-,-) in (5.1) must coincide on the region 

Proposition 8.3. Suppose that 

V?iGN:(,/.o,0i)G5""(r) =^ {x{t,cl)o),y{t,(t>i))eS-^{T), 

(8.4) 

t G [0,00), 

holds. Then for every n G N, we have 

y(<Ao, </>!) = K(<Ao, <Ai) = K+i(</'o, '/'i) = • • • V ((/-o, <Ai) G S~^{T), 
(8.5) 5-"(r) n r = 5-"(r) n r„ = ^-"(r) n r„+i = • • • , 
5""(r) n c = 5~"(r) n c„, = 5~"(r) n Cn+i = • • • . 

Since T and r„ are convex and closed, and S{-, •) is a linear mapping, the 
sets S'~"(r) and S'~"(r„), n G N, are convex and closed. The sets T and r„, 
n G N, are the epigraphs of the continuous functions 7(-) and 7n(")> n gN, 
in Proposition 8.3, respectively. Therefore, 

S-^{T) = {{x,y)GRl:y>S-^Mx)}, 

(8.6) 

S-"(r„) = {{x,y) GRl:y> S-"[7n](x)}, 
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are the epigraphs of the functions S "[7](-) and S "[7n](') for every n G Nq. 
These functions are decreasing, continuous and convex. In fact, 

^.7) 5-"[7](x)=f-^y7('('^Vx|, xeM+,nG: 



and the function 5 ""[771] (■) is obtained by replacing 7 with 7„ in (8.7). The 
support of the functions 'S'~"'[7](-) and 5'~"[7n](-) are 



supp(5-'^[7]) 



supp(5~"[7„]) 



0, 



0, 



/i-1 



respectively, for every n S Z. By Proposition 8.1, the functions S "[7](-) and 
5'~"'[7n](-) are strictly decreasing on their supports. 

Since 5-"[7](0) = (///(/i + 1))"7(0) < 7(0) and 5-"[7](0 > = 7(0 for 
every n € N, the functions «S'~"[7](-) and 7(-) intersect, and 

(8.9) x„(7) = min{xGM+:5'-"[7](a;)= 7(3;)} G (0,00), n £ N. 

Corollary 8.4. Suppose that (8.4) holds. Then for every n = 1,2, .. . 
and k>n, we have x„ = x„(7) = Xni'Jk); o,nd 

(8.10) 5-"(r)nCn([o,x„] xR+) = 5-"(rfc)nCfcn([o,x„] xM+). 

Particularly, we have 7(x) =7„(x) for every x £ [0,x„], and 
V{x, y) = Vn{x, y) V (x, y) G 5-"(r„) n C„ n ([0 



^.11 



n e N. 



Proof. Let us fix any A; > n G N. Since the value functions V{-, •) and 
Vk{-, •) are equal on the region ^""(r) by Proposition 8.3, the boundaries of 
the regions T and coincide in the region S~^{T). Particularly, we have 

(8.12) 7(x) = 7fc(a;) for every x £ [0, Xni^)] 

since 5~"[7](x) < 7(2;) for every x £ [0, x„(7)). Therefore, 

5-"[7](x)=5""[7.](x) 



^.13) 



for every x G 



0, 



Now, (8.12) and (8.13) imply that Xn{'j) =Xni'yk), and (8.6) implies that 

5""(r) n c n ([0, Xni^)] x m+) = s^^iTk) n c, n ([o, xnii)] x m+). 

Equality (8.11) follows immediately from Proposition 8.3. □ 
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The identity in (8.11) suggests that, in a finite number of iterations of 
(7.1), we can find the restrictions of the value function V{-, •) and the con- 
tinuation region C to the set M4. x [B,oo) for any B > 0, when the condition 
(8.4) holds: 

Step A.l. Calculate the value function vi{0,y) for every y G [0,^*], and 
determine7(0) =71(0) =inf{y EM+:ui(x,y) =0} G (0,^); see (8.3), Corol- 
lary 8.4. 

Step A. 2. Given any B > 0, find the smallest n G N such that 

(8.14) ^> (^)"7i(0) = (^)"7n(0) = 5""[7n,](0). 

Because every <S'~™[7m](-)i fri G N, is decreasing, this implies M+ x [B, 00) C 
5-"(r„); see (8.6). We also have n < mm{m £N:B> {n/{p, + !))"'(,*} since 
7i(0)G(0,e). 

Step A. 3. Calculate Vn{(l)o,4>i) for every cpi) G M+ \ D by (7.1), where 
D is as in (6.1). By (8.1), -D C r„ and t;„ = on D. 

Then the value functions V{-,-) and Vn{-,-) are equal on M+ x [B,oo) and 
(M+ X [5, 00)) n C = (M+ X [B, 00)) n Cn. 

The next lemma implies that we can calculate the exact value function 
V{-,-) under condition (8.4) on the set M+ x (0, 00) along an increasing 
sequence of sets M+ x [Bn,oo), and on M+ x {0} by the continuity of the 
function V{-,-) on R^. 

Lemma 8.5. Suppose that (8.4) holds. Let be the same number as in 
the definition of the region D in (6.1). Then HiUtt, )^oo 

S-'^(r) =M+ X (0,oo), 

and 



i.l5) 



1+/X 



r,+oo c5-"(r), ne. 



Proof. Recall from (8.1) that R+ x [^,00] CM.%\D C T. The rectangle 
on the left-hand side in (8.15) is the same set as S'-"(]R+ x [C, 00)) C S'-"(r). 
But, (8.15) implies that R+ x (0, 00) C liminf„^oo5'-"(r). 

On the other hand, for every x G M+, there exists number N{x) such that 
S"^(x,0) = ((1 - l//i)"x,0) ^ r, n > N{x). Then (x,0) ^ ^-"(r) for every 
n > N{x). This implies that limsup„„,oo ^""(r) C M+ x (0, 00). □ 

Remark 8.6. Every set ^""(r), n G N, is separated from its comple- 
ment by the strictly decreasing, convex and continuous function S^^'[y]{x), 
X G [0, in/il-i — 1))"^]- Therefore, the condition (8.4) will be satisfied, for ex- 
ample, if the mappings 1 1— > x{t,(pQ), t G M+, and 1 1— > y{t,(pi), t G M+, are 
increasing for every G M^. We have seen in Section 4.3 that this is 

always the case when A is "large." 
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Thus, if A is "large," then there is a sequence of sets M+ x [Bn, oo), n G N, 
increasing to M+ x (0,oo) in the hmit, such that V{-,-) =Vn{-,-) on x 
[Bn, oo) for every n G N. See also Section 10 below. 

9. The boundaries of the stopping regions. We shall show that the opti- 
mization in (7.1) can be avoided in principle, and vi,V2, - ■ ■ can be calculated 
by integration. 

Note that we obtain Jvn{t, 0Oi 0i) in (7.1) by integrating the function [g + 
fi- S]{-, •) along the curve u i-^ {x{u, 4>o),y(u, cpi)) on n S [0, t]; see (5.3). 
Therefore, the infimum in (7.1) is determined by the excursions oi u 
{x{u, (po),y{u, (pi)), u£ M+, into the regions where the sign of the continuous 
mapping [g + ■ Vn o S]{-, ■) is negative and positive. 

Lemma 9.1. For every n£N, we have 

(9.1) An = {{x,y)eRl:[g + H ■ Vn o S]{x,y) < 0} c: Cn+l ■ 

Proof. Let ((^oi </>!) £ ^n- Since the function u ^ [g + fi-VnO S]{x{u, (j)o), 
y{u, (pi)) is continuous, there exists some t = t{(pQ,(pi) > such that 

rt 

Jvn{t,(pQ,(pi)= / e-^^+^'^lg + ^1- Vno S]{x{u,(Po),y{u,(Pi))du <Q. 
Jo 

Hence, Vn+i{(po,(pi) = JoVn{(po,(pi) < Jvn{t,(po,(pi) < 0, and {(po,(pi) G C„+i. 
□ 

For certain cases, the regions and Cn+i coincide, that is, the contin- 
uation region C„+i for Vn+i{-,-) can be found immediately when the value 
function fn(-,-) is available. Then Vn+i ^ on r„_|_i = \ C„+i, and we 
calculate Vn+i{-,-) on C„+i by the integration 

Vn+l{(pO,(pl) = JVn{t,(pQ,(pl)\t=r„{4>o,,j>^) 

(9.2) = / e-^^+^'^'^lg + ^ivno S]{x{u,(Po),y{u,(Pi))du, 

Jo 

{(po,(pl) G C„+i, 

of the function [g + fi ■ S]{-,-) over the curve {x{-,(pQ),y{-,(pi)) until the 
exit time rn{(po,(pi) [see (5.14)] of the continuous curve u i— > (x(n, (po),y{u, (pi)), 
u G M+ from the continuation region C^+i. 

The region An in (9.1) has properties very similar to those of the contin- 
uation region Cn+i', compare Lemma 9.2 and Proposition 8.1. For example, 
both sets are separated from their complements by a strictly decreasing, 
convex and continuous function which stays flat on the x-axis for all large 
X values. 
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For every n G N, let us define the function a„, : ]R_|_ by 

anix)^mf{y>0:{x,y)eRl\An} 

(9.3) 

= inf{y >0:[g + fiVn o S]{x,y) > 0}. 

Tlie function a„(-) is finite since, given any x £ M+, we have [g + ^ ■ 
S]{x,y) > for every large y G M+. Recall that the function Vn{-,-) vanishes 
outside the bounded region C^^. Th.6 linecir mapping 5:M^ 1-^ in (5.8) is 
increasing in both x and y. The affine mapping g:M?^ i-^ M in (4.12) is also 
increasing and grows unboundedly in both x and y. 

Similarly, given for large x E ]R+ , we have [g + fi-VnO S]{x,y) > 0, V y G M+ . 
Therefore, an{x) = for x € [a,oo) for some q > 0, and 

(9.4) a„ = inf{x > : a„(x) = 0} is finite. 

The set \ An = {{x, y) € '.[g + • S]{x, y) > 0} is convex and 
closed since Vn{-,-) is concave and continuous, S{-,-) is linear and g{-,-) is 
affine. Because \ An is the epigraph of an(-), this implies that On(-) is a 
convex and continuous mapping from M+ into M+. 

The function a„(-) does not vanish identically on R^; in particular, an(0) > 
since the continuous function [g + n • Vn o S]{x,y) is strictly negative at 
(x,y) = (0,0): 

[g + fi-VnoS] (0, 0) = 5(0, 0) + ^ • VniO, 0) < g(0, 0) = -- ^2 < 0. 

c 

Because a„(-) is continuous, this implies that the number an in (9.4) is 
strictly positive. Since ««(•) is convex and vanishes for every large x £ M+, 
it is strictly decreasing on [0,an), and equals zero on [an, oo). 

Lemma 9.2. For n > 1, there exist a number an S (0,oo) and a strictly 
decreasing, convex and continuous mapping an - [0, a„] i— > ]R_^ such that an{an) = 
0, and 

(9.5) {(x, an{x));x £ [0, a„]} = {(x, y) G M^; [5 + // • o S]{x, y) = 0}. 

Moreover, the continuous mapping (x,y) 1— > [5 + • "Wn ° S]{x,y), n > 1, is 
strictly increasing in each argument, and for every n>l, 

{(x,y) G [0,a„) x < a„(x)} 

(9.6) 

= {(x,y) GM^;b + ^-f„o5](x,y) <0} = A. 

Next, we shall relate the regions An in (9.1) and C„_|_i, and their bound- 
aries a„(-) and 7„+i(-), respectively, for every n G N. 
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Using the characterization of the stopping regions r^i; ^ ^ ^5 i^ Propo- 
sition 8.1 in terms of the switching curves 7n.(")) ^^it time in 
Corollary 5.8 can be expressed as 

(9.7) r„((/.o,</)i) = inf{t>0:y(t,</)i)=7„+i(x(t,(/>o))}, (</.o, </>i) G M^, 

since the functions a;(-, 0o) and y{-,(pi) in (4.8) are continuous. Because every 
7„_i_i(-), n S N, is bounded, the function rn{-, •) is real-valued. Thus 

< r„((/)o,0i) < 00 for every {4>o,4>i) G Cn+i- 

Therefore, the (smallest) minimizer rn{4>o,4'i) of the function t Jvn{t, (/>o, (l>i) 
as in (5.13), is an interior point of (0,oo] for every {(pQ,(pi) £ C^+i, and the 
derivative 9Jt;„(t, cpQ, 4>i)/dt vanishes &tt = rn{4>o,4>i)- Using (5.7) and (9.7) 
gives 

0=[g + f^-VnO S]{x{t, 4>o),y{t, (/>l))|i=r„(0o,0i) 

(9.8) = [g + fl ■ VnO S]{x{t,4>o),-fn+l{x{t,(l)o)))\t=r„(^^o,^^), 

{(t>o,(t>l) G C„+i. 

Let us denote the boundary of r„+i by 

(9.9) 5r„+i^{(x,7„+i(x)):xG [0,e„+i]}, 
and define the entrance and exit boundaries of r„+i by 

5r^+i = { {x{rn ((/)o , (Ai ) , <Ao) , 7n+i {y{rn{(l)o,4>i),4>i))), 

for some (0o,0i) G C„+i}, 

(9.10) 

5r^+i ^ {(00, 0i) G r„+i : {x{t, 0o), y(t, ^i)) g c„+i, 

t G (0,(^] for some (5 > 0}, 

respectively. The path t h-> {x(t,(l)Q),y{t,cj)i)) starts at some ((/)o,(/>i) G C„+i 
and enters the region r„+i (for the first time) at the entrance boundary 
dT^^i. Similarly, for every {(j)Q,(pi) G dTf^_^_i, the path t\-^ {x{t,(j)Q),y{t,(j)i)) 
exits r„4.i immediately. 

Remark 9.3. By Lemma 9.5 below, the entrance boundary dTf^_^i is 
a subset of the boundary dAn of the region An in (9.1). Clearly, the curve 
1 1— > {x{t,(j)o),y{t,(pi)) starting at any (tpo^'Pi) G 9r^+i ^ t?^n cannot return 
immediately into the region An [otherwise Jvn{t,(j)o,(j)i) < for some t>0 
and {(j)(),(pi) G C„+i]. In the theory of Markov processes, every element of 
dT^^i (resp. dTn_^_i) is a regular boundary point of the domain A„ (resp. 
the interior of r„+i) with respect to the process 
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Remark 9.4. Observe that for every {(I)q,(I)i) e dT^_^_i, the quantity 
rni4'o,4'i) ill (5-14) is the return time of the curve 1 1— > {x{t,(l)Q),y{t,4>i)) 
to the stopping region r„,_|_i and is also strictly positive. Therefore, the first- 
order necessary optimality condition in (9.8) also holds on the exit boundary 

9r-+i. Thus, 

^=[g + ^l■VnO S]{x{t, (/)o),7n+l(a;(t, <^o)))|t=r„(^o,</>i)' 

(9.11) 

(</.o,<Ai)GC„+iuar^+i. 

9.1. The entrance boundary dT^j^i. Since all of the functions in (9.11) 
are continuous, (9.11) and the definition of the entrance boundary dV^j^i in 
(9.10) imply 

(9.12) [g + ^i-VnoS]{x,y)={}, {x,y)^dTl^^. 

The next lemma follows from (9.12), Lemma 9.2 and the continuity of the 
function [g + ^ ■ S]{-, •). 

Lemma 9.5. For every n G N, let an G M+ and an '■ IR+ ^ IR+ he the same 
as in Lemma 9.2. Then cl(9r^_,_]^) C {(x, an{x)) : x E [0, an]}, n G N. 

Corollary 9.6. For any n G N, if the equality dTn+i = c\{dT'^j^^) 
holds, then 

(9.13) dVn+i = {(x, an{x)) : X e [0, «„]}. 

In other words, £,n+i = On, cind '^n+i{x) = a„(x) for every x G [0,^„+i] = 

(9.14) Cn+l = {{x,y)■.[g + ^l■VnoS]{x,y)<Q:}. 

Proof. By Lemma 9.5, {(x,7„+i(x)) :x G [0,^„+i]} = dVn+i ^ {(x,a„(x)) : 
X G [0,a„]}. Since 7„+i(-) and ««(•) are strictly decreasing, continuous func- 
tions which equal zero at the right-hand point of their domains, they must 
be identical. Finally, 

Cn+i = \ Tn+i = {(x, y) G [0, ^„+i) X M+ : y < -fn+i{x)} 

= {(x,y)G [0,a„) xM+:y<a.„(x)} 

= {(x,y)GlR^ : [g + ^l■VnoS]{x,y)<{)], 

where the last equality follows from (9.6). □ 

If the disorder arrival rate A is large, then every point on the boundary 
dVn+i of the stopping region r„-|_i belongs to the entrance boundary dV^^j^i, 
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see Section 10. Therefore, the stopping boundary dTn+i for the value func- 
tion Vn-^i{-, ■) is determined as in Corohary 9.6, as soon as the value function 
Vn{-,-) is calculated. Using this observation, the main solution method de- 
scribed at the beginning of Section 7 can be tailored into a more efficient 
algorithm; see Section 10 and Figure 4. 

The exit boundary dT^_^_i may not always be nonempty. If it is nonempty, 
it is also determined by the entrance boundary dT^_^i, and the general 
solution method can be similarly enhanced in this case; see Section 11. 

9.2. The exit boundary dV^j^i- Using the semigroup property in (4.9) of 
the functions x{-, •) and •), along with a change of variable, we obtain 

Jvn{t, (Ao, = -e-(^+^)* Ji;„(-t, x{t, (l)o),y{t, ^i)), 

(9.15) 

t>0, {<l)o,(Pi)£Rl. 

Substituting in (5.12) w{-,-) =t;„(-,-), and using the identity above, we ob- 
tain 

JtVn{4>0,4>l) 

(9.16) 

= e-(^+^)*K+i(x(i, (l)o),y{t, (Ai)) - Jvn{-t, x{t, ^o),y{t, <Ai))] 

for t > 0, (<?;>o,(/>i) G M^. Since t;„+i(x(r„((/)o, (/>i), (^o), y('^n(<?^'o, 0i), (Ai)) = 0, 
and J^,^(<^Q^0^)t;„((/)o, (/)i) = i;„+i(0o, (/-i), the equality in (9.16) at t = r„((/)o, 0i) 
gives 

(9.17) Vn+i{cl)o,cl)i) = [-e-(^+^)*J?;„(-t,2;(t,(/>o),y(t,(/.i))]|f=,„(<^o,0i) 

for ((/)o, (Ai) G Recah from Section 9.1 that {x{rn{(t>o,4>i):4'o),y{fn{4'o,4>i)-, 
0i)) G dT^^i for every {(j)o,4'i) G C„+i U ^T%_^_l. Therefore, (9.17) implies 
that we can both calculate the value function Vn+i{-,-) and find the con- 
tinuation region Cn+i by backtracking the curves t^^ (x(— t,(/'o), y{—'t-,4>i)) 
from every point ((/)o,(/'i) G dV^j^i on the entrance boundary. Let us define 
for every ((/"o, (/>i) G M+, n > 0, 

f(</.o, (t>i) = inf{t > : {x{-t, (t>o),y{-t, (/>i)) ^ M^}, 

(9.18) 

rn{(t)o,(t>i) = inf{t G (0,r((/)o,(/)i)] : -Jvn{-t,(l)o,4>i) > 0}, 

where the infimum of an empty set is infinity. Since the mapping t i-^ 
Jvn{t,(j)o,(j)i) is continuous, we have Jvni—rn{4'o,4>i),(po,(pi) = if 
0<r„((^o,<Ai) < OO- 
LEMMA 9.7. The entrance boundary dTf^^i determines the exit bound- 
ary dT^_^_i, the continuation region C„+i, and the value function Vn+ii',-) 
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Fig. 4. Case I: A is "large." The illustration of Method C: Steps C.l and C.2 when (a) 
n = 0, and (b) n — 1 m Step C.l. 
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on Cn+i: 

dV^^^ = {{x{-t,(j)o),y{-t,^i))\t=r„{^o,'Pi) '■ ('^o,0i) G 5r^+i, 

rn{4>o,4>i) < r{4>o,4>i)}, 

C„+i = {(x(-t,(^o),y(-i, </>!)): (</'o,</'i) e 9r^+i,t G (O,r„(0o,</'i)]} \ 5r^+i, 
where f„((/>o, c/)!) = f„((/>o, (/>i) A f(0o, For every (</>o,(/'i) G 9r^+i 

t G (O,f„(0o,(/'i)]. 

10. Case I revisited: efficient methods for "large" post-disorder arrival 
rates. This is Case I on page 13 where A > [1 — (1 + m)(c/2)]+ is "large," 
and the sample-paths of both components of the process ^ = [^P'), $(1)]"^ 
increase between the jumps; see also Figure 1(a). By the relation (4.10), the 
deterministic functions 1 1— > x{t,(j)Q), t S M+, and 1 1— > y{t,(pi), t € M+, are 
strictly increasing for every {(j)Q,(j)i) £ M^. 

By Remark 8.6, the identity in (8.11) between the value functions V{-, •) 
and Vn{-, ■) on the set S~"-(Tn) n C„ n ([0, a;„] x M+) holds for every n G N. 
Thus, we can find the value function V{-,-) by calculating Vn{-,-)i nG'N, 
using steps A.1-A.3 after Corollary 8.4. This method can be improved 
further. We shall show that the optimization in each iteration of (7.1) can 
be avoided, and the value function V{-,-) may be calculated in one pass over 
the continuation region C; see Figure 4. 

Since the boundary dTn+i of the stopping region r„+i is a (strictly) 
decreasing continuous curve, every point in the set dTn+i n int(R2,) IS ac- 
cessible from some point in the continuation region C„+i. Therefore, we 
have dTn+i = cl((9r^^^_|) for every n G Nq. By Corollary 9.6, the set An in 
(9.1) and the continuation region C„+i (and their boundaries) coincide for 
every n gNq. 

If the value function Vn{-, •) = Vn{-, •) for some n G No is already calculated, 
then the boundary of the continuation region C^+i becomes immediately 
available as in (9.13). In fact, (9.5) and (9.14) imply 

(10.1) 5(C„+i) = \{x,y) £ M.l:vnix,y) < 



(10.2) S{dTn+i) = {x, y)£M.l: Vn{x, y) 



' 1 




(2;,y)j 


- ^^ 




' 1 


90S-' 


(2;,y)j 


- 





The set on the right-hand side in (10.2) is a strictly decreasing, convex and 
continuous curve in M^, and it is the same as 

5(9r„_|_i) = 5(the boundary of the set epi(7„+i) n ([0,^„+i] x M+)) 
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(10.3) 



the boundary of the set epi(5[7„+i]) H 



U, 4n+l 



{x,S[jn+i]{x));xe 



0, 



If we know then we can determine the set in (10.2) of aU points 

{x,y) £ satisfying 



(10.4) vnix,y) 



-■goS 



X V X r- 

{x,y) = 7-^ + — ^> 



and obtain the boundary function 7„_|_i(-) after the transformation of this 
set by S~^. Then we can calculate the (smallest) minimizer r„(-,-) of (7.1) 
by the relation (9.7), and the value function Vn+i{-r) by (9.2). We can 
continue in this manner to find the value functions Vn+2{', ")> ''^n+si', • • • • 
This method saves us from an explicit search for the solution rn{4>o,(pi) of 
the minimization problem in (7.1) for every {<j)Q,(pi) £ C^+i: 

Step B.O. Initialize n = 0, vq{-, •) = 0. 

Step B.l. Find the region 



(10.5) Bn = \ix,y)£Rl:Vn{x,y)< 



+ -V2 



/i — 1 /i + 1 c/x 



n> 1. 



Step B.2. Determine the continuation region C„+i = 5~^(i?„) by the 
transformation of Bn under S~^. 

Step B.3. Calculate the value function Vn+i{-, •) on C„+i by using (9.2) and 
(9.7). 

Step B.4. Set n to n + 1 and go to Step B.l. 

In fact, we can do much better than this. After n S N iterations, we find 
both Vn{-, •) and V{-, •), v„+i(-, •), Vn+2{-, ■),■■■ on the subset 

(10.6) Qn = s-"(r„) n c„ n ([o, x„] x m+) 

(10.7) ={(2;,y)G [0,x„] xM+:5-"[7„](2;)<y<7„(x)}, n G N, 
of Cn+i by Corollary 8.4. Therefore, we need to determine only the set 



(10.8) 



Rn+l — Qn+1 \ Qn, 



n G N 



iRi = Qi), 



in Step B.2, and calculate the value function Vn+i{-,-) only on this set in 
Step B.3. By Lemma 8.5, this modified method calculates V{-,-) [and all 
Vn{-, ■), n E N, simultaneously] on any given set M+ x (0, B), B > 0, in a finite 
number of iterations. We shall describe this modified method as Steps C.0~ 
C.2 after Proposition 10.1 after establishing a few facts below. Several steps 
of the method are also illustrated in Figure 4. 
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Since vq = 0, setting n = in (10.4) gives a straight line; substituting 
(x,y) = {x, S[yi]{x)) and comparing this with S{dTi) in (10.3) gives 



(10.9) 



5[7i](x) = -^x + ^.^V2, 
/i — 1 fl c 



X e supp(5[7i]) 



and ^1 = (A/c)\/2. Using (8.7), we find 



(10.10) -/iix)=S-^[S[-/i]]ix)=-x + -V2, xG[0,ei] 



u — 1 A ^' 
0,^f V2 



0, ^V2 

c 



The function 5 ^[7i](-) is affine and intersects with 7i(-) at xi = 2:1(71) = 
(A/c)(V2/2); see (8.9). By Corollary 8.4 and Remark 8.6, the boundary of 
the stopping region on [0,xi] is 



(10.11) 7(2;) =7i(x) =-x+-V2, x£[0,xi] 



0, 



XV2 1 



see the inset in Figure 4(a). Hence, the boundaries of the optimal stopping 
region T and the stopping regions , n G N stick on the upper half of the 
hypotenuse of the rectangular triangle {{x,y) £ :g{x,y) < 0} = cl(Co). 

Proposition 10.1. Fix any n e N. The functions in Sn = {S~^[^k\)'k=i 
do not intersect inside the continuation region = {(x, y) G : y < 7^(2;)}. 
The function S'[7„_|_i](-) intersects with each function in Sn U {7n} pairwise 
exactly once. 



The same conclusions hold when every 7^, /c = 1, . . . , n, in the proposition 
is replaced with 7; this can be verified using the elementary properties of 
convex functions and the affine structure of the boundary function 7(-) in 

(10.11) ; see [2] for the details. Then the proof of Proposition 10.1 follows 
easily from Corollary 8.4. We are now ready to give a better version of 
method B in Section 10 to calculate each v{-,-) and the boundary function 
7(-). Recall that 5'~"'[7n](-) and x„(7„) are defined by (8.7) and (8.9). The 
steps C.l and C.2 below are illustrated in Figure 4 for n = and n = l. 

Step CO. Initialize n = 0, xq = 0, vo{-, •) = and the region Ri as in (10.8). 
Step C.l. Calculate the value function V{(f)o,(f)i) = Vn+i{(po,(pi) for every 
{(po^cpi) e Rn+i using (9.2) and (9.7). 
Step C.2. Set n to n + 1. 

(i) Determine the set 

(10.12) {{x,y) G Rn:vn{x,y) = ^ - + — ^2) 
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of points in i?„ which satisfy (10.4). This is the intersection of the set in 
(10.3) and Namely, it is the section of the strictly decreasing, convex 
and continuous curve x i— > S[^n+i\{x) contained in 

(ii) Find the subset of Rn enclosed between the vertical y-axis and the 
curve in (10.12). This is the intersection ii„ n i?„ of the sets i?„ and 

in (10.5). 

(iii) Find the set Rn+i = S^^{Rn n Bn) in (10.8) by applying the trans- 
formation 5~^(-,-) to the set found in (ii). 

The region Rn+i is enclosed between the y-axis from left, the ^"^-transforma- 
tion of the curve in (10.12) from right. This right boundary of Rn+i ex- 
tends the boundary 7(-) = 7„,_|_i(-) from the previous iteration into the region 

5-(n+i)(r) = 5-("+i)(r„+i). 

(iv) Go to Step C.l. 

11. The smoothness of the value functions and the stopping boundaries. 

The general method described at the beginning of this section evaluates 
the integrals Jvn{-,4>o,4>i) in (7.2) of the function [g + /i- S]{-,-) along 
the curves {x{-,(po),y{-,(pi)) in in order to calculate the value function 
Vn+i{4>Oj4'i) ^ in ('''•I)- For an accurate implementation of this method it 
may be useful to know how smooth the integrand, or essentially the value 
function Vn{-, •), is. 

The smoothness of the value function V{-,-) may also allow us to formulate 
the original optimal stopping problem in (4.12) as a free-boundary problem. 
Then, in principle, we can calculate the value function V{-,-) directly, by 
solving a partial differential equation, as the next proposition suggests. 

Proposition 11.1. Suppose that there is a bounded and continuous 
function u;:R^ (— co,0] which is continuously dijjerentiable on \ dT, 
and whose first- order derivatives are locally bounded near the boundary dT = 
{(x, 7(x)) : X G [0,^]}. Moreover, 

(11.1) {A- X)w{x,y)+g{x,y)=0, (x,y)GC, 

(11.2) w{x,y)=0, {x,y)eT, 

(11.3) {A- \)w{x,y)+g{x,y)>0, {x,y)eT\dT, 

(11.4) w{x,y)<0, (x,y)GC, 

where A is the infinitesimal generator in (A. 4) of the process $ acting on 
the continuously differentiable functions. 

Suppose also that the sample-paths of the process $ = (<!)(*'), <|)(-'^)) spend 
zero time on the boundary dT almost surely, that is. 



(11.5) e: 







lar(*t)dt 



^'0,</'l) G 
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If the convex function ^{■) is also Lipschitz continuous on [0,^,], then w{-,-) = 
V{-,-) on Rl. 

Proof. Similar to the proof of Theorem 10.4.1 in [14], page 215. □ 

Under certain conditions, we are able to show that the bounded, concave 
and continuous value functions Vn{-,-)j ri£N, and V{-,-) are continuously 
differentiable on \ dT'f^^i and \ dV^, respectively, and are not differ- 
entiable on the exit boundaries dT^, and dT^ in (9.10), respectively. The 
exit boundaries dT^, n G N, and dT^ , and the entrance boundaries dT'^, 
and dV^ , are connected subsets of Ml , and we have 



(11.6) ar„ = ar^uci(ar^), nen, and ar = 5r^uci(ar^). 



Moreover, the boundary functions 7n(') and 7(-) are continuously differen- 
tiable on their supports. 

The hypotheses of Proposition 11.1 are satisfied with w{-,-) =v{-,-) in 
(5.10). Thus, the function v{-,-) = V{-,-) may be obtained by solving the 
variational inequalities (11.1)-(11.4). This may be a challenging problem 
since, as we already pointed out above, the smooth-fit principle is guaranteed 
not to hold on some part of the free boundary. We shall not investigate the 
variational problem here, but give a concrete example with this interesting 
boundary behavior and describe our solution method for it. 

The main result is Proposition 11.17 below, and is proven by induction. 
Here, we shall study the basis of the induction by breaking it down in several 
lemmas. The proof of the induction hypothesis is very similar, and we shall 
point out only the major differences after the proposition's statement on 
page 46. 

Let us define the continuous mapping G„ : Mi]_ h-> M by 



((/)o,(Ai) G C„+i,n > 0. 
Using (9.11), (9.12) and Lemmas 9.2 and 9.5, we obtain 

{x(t,4>o),y{t,(l)i))\t=rn{<l>o,4>i) S'9r^+i '^{{x,an{x)):x e [0,a„]} 
(11.9) = {(x, y) G : [<7 + /i • o S] (x, y) = 0}, 

(</.0,(/>i)GC„+iU5r-+i. 



G„,(t,(/)o,</'i) = [g + fi-Vn.o S]{x{t,(j)o),y{t,cl)i)) 



(11.7) 



(t,(/.o,(Ai)GM3,n>0. 



Note that (9.2) gives 



(11.8) 
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Therefore, for every n G N, 

(11.10) o = G„(t,(/)o,(/)i)|t=r„{0o,<Ai)' (0o,</'i) GC„+iu9r^+i. 

Under certain local smoothness and nondegeneracy conditions on the 
function the implicit function theorems guarantee that the equa- 

tion Gn{t,(po,4'i) = determines t = tn{(l)o,4>i) in an open neighborhood of 
every point {rn{(l)o,4>i),(j)o,(j)i) in M+ x C„+i, as a smooth function of the 
variables ((^Oji^i)- Since the continuation region C„_|_i has compact closure, 
a finite subcovering of these open neighborhoods exists. Patching the so- 
lutions tn{-,-) in the finite subcovering gives the global solution, which is 
smooth and must coincide with r„(-,-) on Cn+i- 

In the remainder, we shall use the following version of the implicit function 
theorem; see [16], Chapter 14, and Conjecture 11.18. 

Theorem 11.2 (Implicit function theorem). Let A C M™- be an open set, 
F:^ M be a continuously differentiable function, and {t,x) G R x R'""^ 
be a point in A such that 



d 

F{t,x) = and —F{t,x) 



{t,x)={t,x) 



Then there exist an open set B C R'""^ containing the point x, and a 
unique continuously differentiable function f:Bh^R, such that i= f{x) 
and F{f{x),x) = for all x £ B. 

Since vo{-, •) = 0, we have 

Go{t, (1)0, 4>i) = x{t, 4>o) + y{t, (/)i) - -V2, 

(11.11) 

(t,(/.o,(Ai) GR+ X Ci. 

The function Gq{-,-,-) is continuously differentiable on M+ x Ci. By (6.8) 
in Section 6, its partial derivative 

(11.12) DtGoit, 00, 0i) = j^[x{t, M + yit, cPi)] 

with respect to i-variable may vanish at most once; if this happens, this 
derivative is strictly negative before, and strictly positive after, the derivative 
vanishes; otherwise, it is strictly positive everywhere (see also Figure 2). 
Namely, the function t Go{t,4>o,(pi) has at most one local minimum for 
every (0o,</'i) G 

Lemma 11.3. Fix any ((j)Q,(pi) £M?^_. The function t^^ Go{t,(l)Q, (pi) from . 
into R has at most one local minimum. It is strictly increasing if there is no 
local minimum. If there is a local minimum, then the function Go{-,(f>o,(f)i) 
is strictly decreasing before the minimum and strictly increasing after the 
minimum. 
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Lemma 11.4. The smallest minimizer rQ{(j)Q,(j)i) in (5.13) is continu- 
ously differentiable at every {(j)o,(f)i) G Ci. 

Proof. The result will follow from Theorem 11.2 applied to the function 
Go(-, •, •) on M X Ci at the point {t,x) = (ro (00, 00; 0i) G K x Ci. We only 
need to estabhsh that 

DtGo{t,(pQ,(pi)\t=ro{4>o,<Pi) {4'o,4>i) G Ci. 

Let us fix {(j)Q,(j)i) £ Ci and assume that DfGo(?'o(0O5 ^i), 00, 0i) = 0. 
Then the function t Go{t, cpQ, cpi) is strictly decreasing on t G [0, ro(<^05 ^Pi)] 
by Lemma 11.3 and 

Go(t,(/>o,(/>i) > Go{ro{(/)o,(/)i),(j)o,(l)i) =0, te [0,ro((/)o,</'i))- 

Therefore, (11.8) implies that fi(0O)0i) > 0. This contradicts our choice of 
{(j)Q,(pi) in the continuation region Ci as well as the bound vi{-, •) < 0. □ 

Corollary 11.5. The value function vi{(j)Q,(f)i) is continuously differ- 
entiable at every (</)o,i;^i) G Ci. For every {(j)Q,(j)i) G Ci, 

^'i(</'o, </>!)= / e-^^+^^'D^,Go{t,<Po,<Pi)dt 
Jo 

I — g-{/i-l)»'o{0o,<^i) 

(11.13) 

D^,vi{<Po,(t>i)= / e-^^+^'^'D^^G^{tAoAi)dt 
Jo 

I _ g-(/i+l)'-o(0o,</'i) 
~ Ai+ 1 ■ 

Proof. By (11.8) and Lemma 11.4, the value function wi(-,-) is con- 
tinuously differentiable. Using (11.9) after applying the chain-rule to (11.8) 
with n = gives the integrals in (11.13). These integrals can be calculated 
explicitly by using (4.7) or (4.8). □ 

Corollary 11.6. The entrance boundary dTf in (9.10) is connected. 
More precisely, 

(11.14) 5r^ = {(x,7i(x)):xG(ef,ei)} /or some < < 6, 

where is the same as in [0,^i] =supp(7i), the support of the boundary 
function 7i(-); see Proposition 8.1. 
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Corollary 11.7. The restriction of the boundary function to the 
interval (Ci)Ci) continuously differentiable. In fact, 



71 (x) =aoix),xe [^1,6], [0,6] = supp(7i) = supp(ao) = [0,ao], 



is the continuously differentiable boundary function of the region Aq = {{x,y) £ 
Rl :[g + fi-VQO S]{x,y) <0} in (9.6). 

Proof. The function ao(-) in (11.15) is continuously differentiable on 
its support supp(ao) = [0,ao]) and the result follows from Lemma 9.5 and 
Corollary 11.6. □ 

The entrance boundary dTI always exists. However, the exit boundary 
dTf may not exist all the time. Next we shall identify the geometry of the 
exit boundary dTf when it exists. 

Lemma 11.8. For every {(f)Q,(f)i) G dTf, we have [g + fi ■ vqo S]{(f>o,(j)i) > 
0. Therefore, cl{dTl) D dTf = . 

Proof. Suppose that (^Oj^i) G dTf. Let us assume that [g + n • vq o 
S]{4>o,4>i) < 0. Then Go(0, (/>o, 0i) = [g + iJ,-vooS]{(po,4>i) < = Go{ro{4>o,(pi), 
<j)o,4>i), and Lemma 11.3 implies 

Go(t,(/>o,'Ai) = [g + fJ' ■ vq o S]{x{t,<j)o),y{t,(j)i)) <0, t E (0, ro((/)o, </'i))- 

This inequality and (11.8) for n = give 



which contradicts i;i(0o5 0i) = 0- This proves that [g + ^-voo S'](0O5 0i) > 
for every {(pQjfpi) G dTf. Since the mapping {x,y) [g + n • vq o S]{x,y) is 
continuous, we have [g + fi-VQO S] (0O) 0i) = for every {cpo, cpi) G cl(5r^) by 
Lemmas 9.2 and 9.5. Therefore, cl(9rf ) n dTf = 0. □ 

The next corollary is helpful in determining the point (Cb7i(Ci)) = (?i) 
ao(^f)). The region An was introduced in Section 9. 

Corollary 11.9. The parametric curve 



where 



(11.15) 





(11.16) 
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is the smallest among all parametric curves M?^ri{{x{t,(j)o),y{t,cl)i)):t ^W}, 
{(po,(pi) € that majorize the boundary function ao(-) of the region Aq = 
{{x,y) -.[g + n-voo < 0} = {(x, y) G : y < ao{x)}. 

The curve C\ and the boundary OAq = {{x,ao{x)) :x E [0,q;o]} touch ex- 
actly at ('^i,«o(^f)) = (^f)7i(^f)) <^f^d nowhere else. 

Proof. By Corollary 11.7 and Lemma 11.8, we have (^f,ao(^f)) = (^f, 
71 G c\{dTi) \ dTl. Therefore (Cf, 00(^1)) i dVl U dVf. Hence there ex- 
ists some 5 > such that 

(11.17) (x(t,e?),2/(t,ao(^?)))GriC]R^\Ao, te{-5,+5). 

Recall from (9.18) that '^(Ci; oo(Ci)) is the exit time of 1 1-^ {x{—t,^^),y{—t, 
ao(^i))) from M^. Then the function 

t ^ Goit,ei,aoiei)) ^[g + fi-VQO S]{x{t, ei),y{t, ao(^O)), 

tG[-r(ef,«o(ef)),oo) 

has a zero at t = and is nonnegative for every t S (—5,5) by (11.17). 
Hence it has a local minimum at ('^f, ao(Ci))- Lemma 11.3, the func- 
tion Go(t,^f , ao(^f )) is strictly positive for t^O. Therefore, y{t,ao{S,i)) > 
ao(x(t,^f)) for t /O. □ 

Remark 11.10. Since Ci c \ Aq, we have JvQ{t, (po, ^i) > for t > 
and ((^Oi'/'i) G Ci- This implies vi{(j)o,(pi) =0 for {(j)Q,(pi) £ Ci. Therefore, 
CiCTi. 

The curve Ci divides into two components. Since the continuation 
region Ci is connected and contains Aq, the region Ci is contained in the 
(lower) component which lies between Ci and x-axis. Thus the boundary dTi 
is completely below the curve Ci, and they touch at the point (Cf,ao(Ci)) = 
(Cf,7i(ef)). See Figure 5(a). 

The next corollary shows that no point on the boundary {(x,ao(x)) ix £ 
[0, )} over the interval [0, ) of the region Aq = {{x, y) G -.[g + ^ ■ vq o 
S]{x,y) < 0} is a boundary point for the stopping region Fi. 

Corollary 11.11. For every x G [0,^f), we have 71 (x) > ao(x) and 
[g + li-VQoS]{x,-ii{x)) >0. 

Proof. If [0,^f) = 0, there is nothing to prove. Otherwise, fix any (j)Q G 
[0,^f). Assume that {4>Q,aQ{4>Q)) E dFi. By Corollary 11.6 and Lemma 11.8, 
we have {(j)Q,aQ{(j)Q)) ^ dTl U dVf. The same argument as in the proof of 
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Fig. 5. (a) and (b) illustrate Steps D.l and D.2 of Method D /orn = 0, and (c) and (d) 
/or a general n. 



Corollary 11.9, with (c^O; '3^o(0o)) instead of (Ci) ^^0(^1))) gives that the para- 
metric curve {(x(— i, (/iq), y(— ao(0o))) : * £ [— ^(<?i'0) ao('^o)); 00)} is the small- 
est majorant of the boundary function oo(-)) and both curves touch at the 
point (^0) Q^o(<^o))- But this implies 4>o = ^i, a contradiction with our choice 

of (t)Q. □ 

Corollary 11.12. //(/>o G [0,^f), then (</'o,7i(</'o)) G dVf has an open 
neighborhood, on the intersection with the continuation region Ci of which 
the function ro(-, •) is hounded and hounded away from zero. 

On the other hand, the function ro(-,-) is continuous on the entrance 
boundary dT\: for every {(j)Q,(f)i) G SFf and every sequence {(0q"\ ^ 
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Ci converging to the boundary point ((/jq, <^i), we have lim^^oo ^o(</'o ' '^i ) ~ 
0. 

Lemma 11.13. // = 0, then dTi = d{dTl). If > 0, then the exit 
boundary dTf is not empty, and dTi = dTf U cl(5rf ). 

If dTi 7^ cl(9rf), then > 0, and the exit boundary dTf is not empty 
by Lemma 11.13. The characterization of the exit boundary in Lemma 9.7 
can be cast as 

rf = {(x(-t,(/.o),y(-t,ao(</'o)))|t=fo(</.o,ao(</'o)) ^-^o G 
for some G (■^fi^i)- More precisely, 

(11.18) ef = mf{0o G [ei,6] :ro(</.o,7i(a^)) < r{^o,7i{x))}- 

For every 0o G we have fo((/)o, 7i('/'o)) < ^((/"o, 7i(0o)- See (9.18). 

Our next result shows that the exit boundary dTf = {(0Oi 7i(0o)) : i^o G 
[0,i^f)} is on a continuously differentiable curve, if it is not empty. 

Lemma 11.14. The restriction of the boundary function 7i(-) to the 
interval [0,^f) is continuously differentiable. 

If the value function vi{-,-) were continuously differentiable on the exit 
boundary dTf, then the result would follow from an application of the im- 
plicit function theorem to the identity vi{(j)o, (pi) = near the point (00, (/>i) = 
((/)o,7i(</>o))- Unfortunately, vi{-, •) is not differentiable on dTf; see Lemma 11.16. 
Therefore, we shall first extend the restriction to the set Ci U dTf of the 
value function ui(-, •) to a new function wi(-, •) on an open set i?i D Ci U dTf 
such that ^i(-, •) is continuously differentiable on Bi. We shall then use the 
identity vi{(j)o,'yi{(j)i)) = as above. 

Lemma 11.15. The boundary function 7i(-) is continuously differen- 
tiable on the interior of its support [0, ^i]. 

The next result shows that the value function is not differentiable on the 
exit boundary dTf. In fact, as the proof reveals, the left and right partial 
derivatives are different along the exit boundary. Therefore, the smooth-fit 
principle does not apply to the value function vi{-,-) along (some part of) 
the boundary, if the exit boundary dTf is not empty. 

Lemma 11.16. The value function vi{- , ■) is continuously differentiable 
on the entrance boundary dTf , but is not differentiable on the exit boundary 
dTf. 
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The techniques used above in the analysis of the value function vi{-, •) and 
the boundary function 7i(-) can be extended by induction to every function 
Vn{-,-) and the boundary function 7n(')) if the following are true for every 
n G N: 

Ai(n): For every (0o, 0i) G M.\, the function t G„(t, 4>o,4>i) in (11.7) from]R+ 
into M has at most one local minimum. It is strictly increasing if there 
is no local minimum. If there is a local minimum, then the function 
Gn{-,4'Q,4>i) is strictly decreasing before the minimum and strictly 
increasing after the minimum. 

A2(n): The function ^ [g + ^ ■ Vn ° S]{x,y) is (continuously) differ- 

entiable on the entrance boundary 9r^_|_^ of the stopping region 
r„+i = {{x, y) : Vn+i{x, y) = 0}. 

Proposition 11.17. If Ai{k) and A2{k) above are valid for every < 
k <n, then the following hold: 

1. The value function Vn+i{-, •) is continuously differentiable on \ T^^i 
everywhere except the exit boundary dV^^j^i. For every ((/)o,</'i) G Cn+i, 



X {x{u,<j)o),y{u,(t)i))du. 

2. The entrance boundary dV'^_^_]^ is connected. More precisely, 

5r^+i ={(x,a„(x)):j;E (^^+i,C„+i)} for some G [0,Cn+i). 

The boundary function an{-) of the region An = {{x, y) G : + /i • i;„ o 
S]{x,y) < 0} is continuously differentiable on (^^_|_;^,^n+i)- Therefore, the 
boundary function 7„+i(-) = a„(-) on (^^_|_i,^n+i) is continuously differ- 
entiable. 

3. The parametric curve 





X {x{u,(l)o),y{u,(l)i))du 



and 





Cn+l = n {(X(t, en+l).y{t, an{en+l))) -t 
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is the smallest among all the parametric curves M?^n{{x{t, (j)o),y{t, : t E 
M}, (00)1^1) £ tt^CLt majorize the boundary function a„(-) of the region 
An = {ix,y):[g + n-vno S] {x,y) <0} = {{x,y) €Ml:y < a„(a;)} . 

The curve Cn+i and the boundary dAn = {(x,a„(x)) :x E [0,a,i]} touch 
exactly at an(C^+i)) = (^n+i,7n+i(^^+i)) and nowhere else. 

4. If = 0, then dTn+i = cl(5r^_,_]^). If > 0, then the exit boundary 
dTfi^_^_i is not empty, and dVn+i = dT^_^_i U cl{dT^_^_i). 

5. The boundary function 7„+i(-) is continuously differentiable on the inte- 
rior of its support [0,^„+i]. 

The proof of the proposition is by induction on n E Nq. The suppositions 
Ai(0) and A2(0) are always vahd; see Lemma 11.3, and note that [g + fi ■ 
vqo S]{- , ■) = g{- , ■) is continuously differentiable everywhere. All of the claims 
are proved for the basis of the induction n = before the statement of the 
proposition. For n > 1, the proofs are the same with obvious changes, with 
the exception of the differentiability of a„(-) in part 2 of the proposition. 

For n = 0, the differentiability of ao{x) = —x + (A/c)\/2, x E (0,^i) was 
obvious. For n > 1, the function a„(-) is not available explicitly, only through 

[g + fi-VnO S]{x,an{x)) = 0, XE [0,^n+l]- 

By A2(n), the function [g + fj, ■ Vn ° S]{-,-) is continuously differentiable 
on ar^+i = {(x,an(x)) :x E {Cn+i,Cn+i)}- Since y^ [g + fi ■ Vn o S]{x,y) is 
strictly increasing for every x E , we have 

d 

— [5 + /i-Uno5'](2;,y)|(^^y)=(^^a„(^)) > 0, xE (C+iiU+i)- 

Thus, the function a„(-) is continuously differentiable on ^n+i) by the 

implicit function theorem. 

11.1. The interplay between the exit and entrance boundaries We have 
been unable to identify fully all the cases where the hypotheses Ai{n) and 
A2(n) on page 46 are satisfied for every n E N (see, though. Section 11.2 
for the important case of "large" disorder arrival rate A, and Section 11.3 
for another interesting example, where these hypotheses are satisfied). How- 
ever, they are sufficient for Proposition 11.17 to hold, and Proposition 11.17 
shows the crucial interplay between the exit and entrance boundaries. We 
would like to illustrate this interplay briefly; it may be very useful in design- 
ing efficient detection algorithms for general Poisson disorder problems. We 
suggest how the gap may be closed as an interesting open research problem. 

In Section 9.2, we showed that both the value functions and the exit 
boundaries are determined by the entrance boundaries; see Lemma 9.7. More 
explicitly, once the entrance boundary dV^j^i has been obtained, one can 
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calculate the value function Vn+i{-, ■) and the exit boundary dT^_^_i by run- 
ning backward in time the parametric curves t {x{t,4>o),y{t,(j)i)) from 
every point {(pQ,(j}i) on the entrance boundary dT^_^_i and by evaluating the 
explicit expressions of Lemma 9.7 along the way. On the other hand, the 
entrance boundary dT^^^ can be found when the value function Vn{-, •) has 
already been calculated. Since f o = is readily available, the following itera- 
tive algorithm will give us every ?;„(•,•), n € Nq, and the boundary functions 
7„(-); see also Figure 5. 

Step D.O. Initialize n = 0, vo{-,-) = on M^. Let ao(-) be the boundary 
function of the region Aq = {{(j)Q,(/)i) £ : [g + fi ■ vq o S]{(j)o,4>i) < 0}; see 

(n.i5). 

Step D.l. There is unique number (j)Q = Cn+i ™ the bounded support 
4>o £ [0,^„+i] of the function a„(-) such that, for every small 5 > 0, we have 

an{x(t,(l)o)) <y{t,an{(l)o)), t£[0,6) if (/>o = 0, and 

te{-6,6) if(/.o>0. 

Equivalently, the parametric curve Cn+i — n {{x{t, ^^^i),y{t, an{Cn+i))) '■ 
t G M} in (3) of Proposition 11.17 majorizes the boundary {{x,an{x)) :x G 
supp(art)} of the region An = G : -h ^ • o S]{x,y) < 0} ev- 

erywhere. The entrance boundary of the stopping region r.„+i = {{(pQjCpi) £ 
Rl:Vn+i{(t>o,(t>i) = 0} is givenby 5r^+i = {(0o,a„((?:>o)):(Ao G {C+i,^n+i)}- 

(i) Find the entrance boundary dT^^i . 

(ii) For every ((/)o,</'i) G dT^^i, take the following steps to calculate the 
value function Vn+i{-, •) on the continuation region C„+i and the exit bound- 
ary 9r^+i: 

(a) Calculate f((/>o, = mf{t > : {x{-t, 4>o),y{-t, 4>i)) ^ M.I}. 
(i) If — Ju„(— r((^05 <Ai)5 '/'Oi < 0, then set rn{4>Q,4>i) = oo. Otherwise, 
find 

rn{(l)o,4>i) - inf{t G (O,r(0o,0i)] : -Jvn{-t,(j)o,4>i) > 0} 
by a bisection search on (O,r(0O)0i)]) and add the point 

{x{-fn{4>o,4'i),cl)o),y{-rn{(po,(l)i),4'i)) S dV^+i 

to the exit boundary. 

(c) Calculate the value function 

Vn+l{x{-t, 0o), y(-i, (t>l)) = -e-'-^+f'^'jVni-t, (^Q, 

along the curve (x(-t,(/>o),y(-t, </>!)), t£ (0, f((/)o, (/>i) A f„((^o, <Ai)], until it 
either leaves or hits the exit boundary dV^^i. 

The union ar^+i Ucl(ar^+i) = ar^+^U {(x, an{x)) : x G [Cn+i,Cn+i]} gives 
the boundary dTn+i = {(x,7n+i(x)) :x G [0,^n+i]} and the boundary curve 
7n+i(-); which is strictly decreasing and convex on its support [0,.^„+i]. 
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(iii) Set7;„+i(-,-) = on the stopping region r„+i = {(x, y) : y > 7„+i(x)}. 
Step D.2. Set n to n + 1. Determine the locus of the points {(f)Q^(f)i) in 



that satisfy the equation 



1 



This locus is the same as {(x, S'[a„+i](x)) : x G supp(5'[a„+i])}; see Nota- 
tion 8.2. Shift it by the linear transformation of (5.8) to obtain the 
boundary {(x, a„(x)) : x G supp(on)} of the region An = {{x,y) -.[g + • Vn o 
5](x,y) <0}. Go to Step D.l. 



Conjecture 11.18. The algorithm relies on only two results from Sec- 
tion 11: (i) the entrance boundary dT^^i, n £ Nq, is connected, and (ii) the 
boundary dT^^i, n S No, is the disjoint union of the exit boundary dT^j^i 
and the closure of the entrance boundary 9r^^_|. Part (ii) was proved by 
using (i) and the first hypothesis Ai(n + 1) on page 46; see Lemma 11.13. 
We conjecture that the hypothesis Ai(n + 1) always holds for all n G Nq. 

On the other hand, part (i) was proved by using the continuity of the map- 
ping (00, (/>i) > r„(</)o, (/>i) on the connected continuation region Cn+i; see 
Corollary 11.6. The continuity of the mapping rn{-,-) followed from its con- 
tinuous differentiability on Cn+i, which we proved by using the implicit func- 
tion theorem (Theorem 11.2) under hypothesis A2(n + 1); see Lemma 11.4. 
We conjecture that this mapping is always continuous on the continuation 
region C„+i. This may be proved directly by using a weaker version of the 
implicit function theorem (see, e.g., \12\) or by using nonsmooth analysis 
(see, e.g., [7]). 

11.2. The regularity of the value functions and the optimal stopping bound- 
aries when the disorder arrival rate A is ^Harge." One of the cases where 
both Ai(n) and A2(n) on page 46 are satisfied for every n £ N, is when the 
disorder arrival rate A is "large;" see Section 4.3 and Figure 1(a). 

Suppose that A > [1 — (H-//)(c/2)]"'". Then the curve t (x(t, 4>o),y{t, (pi)), 
and therefore the mapping 1 1-^ Gn{t, (pOjCpi), t £ M+, are strictly increasing 
for every {(pQ,(pi) E and n £ Nq; see Lemma 9.2. Hence, Ai(n) always 
holds for every n G Nq. 

For the same reason, all of the exit boundaries dT^, n G N, are empty; 
see Section 10. Since dTf is empty, the value function vi{-, ■) is continuously 
differentiable everywhere. Therefore, A2(l) holds. Then Proposition 11.17 
implies that V2{-,-) is continuously differentiable everywhere since c^Ff is 
empty. Therefore, A2(2) holds, and so on. 

Corollary 11.19 ("Large" disorder arrival rate: smooth solutions of 
reference optimal stopping problems). Suppose thatX> [1 — (l + /.f)(c/2)]+. 
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Then Ai{n) and A2{n) hold for every n (zNq, and Proposition 11.17 applies. 
Particularly, for every n G Nq 

1. The value function Vn+i{- , ■) is continuously dijjerentiable everywhere. 

2. The exit boundary dTfi^_^_i is empty, and dTn+i =cl{dT'^^i). 

3. The boundary function 7„-|_i(-) is continuously differentiable on the inte- 
rior of its support [0,^„+i]. Thus, the function 7„_|_i(-) coincides with the 
boundary 

\V2^ 



ao(x) = —X H — \/2 of the region Aq on the interval 
c 



0,- 



2c 



and fits smoothly to this function at the right end-point of the same in- 
terval. 



The last part of (3) in the corollary follows from (10.11) in Section 10 
and Proposition 11.17. Recall also from Remark 8.6 that, if the disorder 
arrival rate A is "large," then there is an increasing sequence of sets M+ x 
[i?„,oo) whose limit is IR+ x (0,oo), and =Vn{-,-) on x [i3„,oo) for 

every n G N. Therefore, Corollary 11.19 implies immediately that the value 
function v{-, •) and the boundary function 7(-) are continuously differentiable 
on R_|_ X (0,oo) and on the interior of the support [0,^] of the function 7(-), 
respectively. 

To prove that v{-,-) is continuously differentiable on (0,oo) x {0}, we 
shall use again the implicit function theorem. By Proposition 5.6 and Re- 
mark 5.10, 

rri4>afl) 

v{^o,0) = Mr{^o,0),^o,0)= e-(^+^)*G(t,(/.o,0)a!t, </.o G M+. 

Jo 

The function (t, (/>o) ^ (/>o,0) = [g -\- fi ■ v o S]{x{t,(j)o),y{t,0)) is con- 
tinuously differentiable on (0, oo) x (0, oo) since v{-,-) is continuously dif- 
ferentiable on M_|_ X (0, oo) and (x(t, (/jq), 0)) E (0,oo) x (0, cxo) for ev- 
ery t > 0. Moreover, the partial derivative (t, 0o) '-^ Dtj)oG{t,(l)o,0) is lo- 
cally bounded on (0, cxd) x (0, cxo) by Corollary 5.4. Therefore, the function 
(t,(/)o) I— > Jf(i,(/>o,0) is continuously differentiable on (0,cx)) x (0,oo) and 

D^,Mt,4>o,0) = re-(^+'^)"Z)<^„G(t,(/.o,0)(i^x 
Jo 

= I" + _ i)D^^y o S] (x(n, </.o), 0)) du, 

Jo 

{t,^o)£R+ X (0,oo). 

Since v{cj)o,0) = for every cpo £ oo), it is continuously differentiable on 
(^, oo). To show that it is differentiable on (0,.^), it is enough to prove that 
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the mapping (pQ r{(j)Q,0) from (0,^) to M_|_ is continuously differentiable. 
Observe that, if we define 

F{t, (ko) = l{x{t, (t>o)) - y{t, 0), (t, (/>o) e M^, 

then F(r(</)o,0), (po) = for every 0o £ [0,C]- Foi' every (j)Q G (0,^), the func- 
tion F{-, •) is continuously differentiable in some neighborhood of (ro(0o5 0), (/>o) 
since x{r{(j)Q,0),(po) £ (0,^) and 7(-) is continuously differentiable on [0,^). 
Moreover, 

DtF{t, (l)o) = y ^o))Dtx{t, 0o) - Dty{t, 0) < 

at every {t,cl)o) G where DtF{t,<j)Q) exists [because since 7(-) is decreas- 
ing, t x{t,0) and 1 1-^ x{t,(j)Q) are strictly increasing]. Then the implicit 
function theorem implies that cpo i-^ r{(j)Q, 0), and therefore, (po i— > v{4>o,0) = 
Jv{r{(j)Q,0),(pQ,0), is continuously differentiable at cpo G (0,^). An argument 
similar to that in Corollary 11.12 shows that (po ^ r{(f)Q,0) is continuous at 
00 = ^ and lim^yig r(i;^0) 0) = 0. By the Leibniz rule (see, e.g., [16], Theo- 
rem 11.1, page 286), the limit of the derivative 

D^,v{cPo,0) = DtJv{r{4>o, 0),(po,0) + D^, Mr (^0,0), ^o,0) 

[1 + _ l)D^,v o S] (x(n, <^o), 0)) du, 

00 G (0,0, 

of the value function v{-,-) at {(po,0), as cpo increases to ^, equals zero. Recall 
that, since r(0o,O) > for every cpo G [0,^, the derivative of t Jv{t,(/)Q,0) 
on the right-hand side vanishes at its minimizer t = r((pQ,0). Thus, the left 
and right derivatives of the concave function 0o ^(00? 0) at 4>o = ^, are 
equal: 

^^o^(^' 0) = 1™ ^^o^('^0' 0) = 1™ ^*o«(0o, 0) = = t;(e, 0). 

ru ^^^^ v-u ^jjI^ 

This shows that the function (0,oo) 3 (pQ v{(po,-) is continuously differ- 
entiable. Hence the value function v{-,-) is continuously differentiable on 
M+ X {0}. 

Corollary 11.20 ("Large" disorder arrival rate: smooth solution of the 
main optimal stopping problem). Suppose that A > [1 — (1 -|- fi){c/2)]~^ . 
Then: 

1. the value function v(-,-) is continuously differentiable everywhere; 

2. the boundary function 7(-) is continuously differentiable on the interior 
of its support [0,1^] , coincides with the boundary function 



uqIx) = —X H — \/2 of the region Aq on the interval 
c 



0,- 



2c 
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and fits smoothly to this function at the right end-point of the same in- 
terval; 

3. the function v{-,-) is the solution of the variational inequalities (11.1)- 
(11.4). 

Corollary 11.21. // A > [1 - (1 + ^)(c/2)] + , then both the sequence 
{vn}neN of the value functions and the sequences of their partial derivatives 
{DfiyQVn}neN o,nd {D^^Vn}n&i Converge uniformly to the value function v 
and its partial derivatives D^^^v and D(j,^v, respectively. 

Proof. This follows immediately from Theorem 25.7 in [17], page 248. 

□ 



The results obtained in Section 9 for the functions Vn, n G N can be 
extended easily to the value function u(-,-), n G N. As in Lemma 9.2, we 
have 

A ^ {(x, y)^M'^■.[g + ^i■voS][x,y)<Q} = {[x,y)e^l■.y< a(x)}, 

{{x,y)em.\■.[g + ^i■vo 5](x, y) = 0} = {(x, a(x)) : x G [0, a]} 

for some decreasing function a : M4. ^ M+ which is strictly decreasing on its 
finite support [0,a]. We have yl C C, and equality holds if A > [1 — (1 + 
//)(c/2)]+ since the parametric curves t^ {x{t,(j)Q),y(t,4>i)) increase and do 
not come back to the region A after they leave; see Section 10. Therefore, 
7(-) = a(-) and 



(11.19) 



[g + f,-voS]ix,y)>0, {x,y)Gr\dT. 



Proof of Corollary 11.20. Only (3) remains to be proven. The 
function v : 1-^ (— oo,0] is bounded and continuously differentiable. By 
the definition of the continuation region C = {[x,y) G : v{x,y) < 0} and 
the stopping region T = \ C, the (in)equalities (11-2) and (11.4) are 
satisfied. On the other hand, (11.19) implies 

[{A - \)v + g]{^o,(l)i) = [9 + 1^ ■ V o S] {(t>o, {4>o,(l)i)& r, 

is strictly positive for every (cpQ, cpi) G F \ dT, that is, (11-3) is also satisfied. 
On the other hand, [{A — X)v + g]{(l)o,(pi) equals 



(A + 1) 



A(l 



m 



V2 



A(l + m) 



1 

1 - - 

IJ- 



1 

1 + - 

IJ- 



4>i -v{(po,4>i) 



V2 

\v{(l)o,(j)i) g{(j)o,(j)i) 



D^ov{(l)o,(t)i) ■Dtx{0,(t>o) 
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Jo 



t=0 




= -^Ae'^^^^^^'vixit, 4>o),y{t, (/.i)) + Jv{t, 00, (/'i)]|i=o, {^o, (ki) G C. 



Observe that the expression in the square brackets of the last equation equals 
v{(j)o,(j)i) for every sufficiently small t>0, by (5.19) in Remark 5.10. There- 
fore, the derivative above equals zero, and (11.1) holds. This completes the 
proof that the function v{-,-) satisfies the variational inequalities (11.1)- 



The boundary function 7(-) is strictly decreasing on its support. The 
process $ can have at most countably many jumps, and its paths are strictly 
increasing between the jumps. Therefore, the time that the process $ spends 
on the boundary dF = {{x,^{x)) :x G [0,^]} equals zero almost surely. Since 
the derivative of the convex boundary curve > 7' (2;) > 7'(0+) = ao(0+) = 
— 1 is bounded on x G (0, ^), the curve is Lipschitz continuous on its support. 



Finally, Corollary 11.20 also shows that for every A > [1 — (1 + m)(c/2)]+, 
the smooth restrictions of value function Vn+i{-, •) to the continuation region 
Cn+i and to the stopping region r„+i fit to each other smoothly across the 
smooth boundary dTn+i = {(x, 7„+i(x)) : x G [0,^n+i]}- 

However, if < A < 1 — (1 + m)(c/2) is small, then the corresponding value 
function does not have to have the same smooth-fit property. 

11.3. Failure of the smooth-fit principle: a concrete example. Here we 
shall give a concrete example of a case where the value function fits smoothly 
across the entrance boundary, but fails to fit smoothly across the exit bound- 
ary of the optimal stopping region; see Figure 6(d). 

Suppose that the disorder arrival rate A, the pre-disorder arrival rate 
/i of the observations, the detection delay cost c per unit time, and the 
expectation m = Eo[A — /x] of the difference A — between the arrival rates 
of the observations after and before the disorder, are chosen so that 



(11.4). 



□ 



0< A< l-(l + m)(c/2), 



// + 1 



(11.20) 
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y < S[ao]{x) = ^^-^ao (—^x 

Here (f>d > is the mean-reversion level in (4.14) of y i-^ y{t,(f)i) for every 
initial condition (pi E see Section 4.4. The point 




Fig. 6. (a) shows the location of points {4'oj4'i) '^^d. (0,^i) and the line described by 
the function S[ao](-). In (b) and (c), we recall how to find the function Vi{-) and region 
Ai, respectively; compare with Figure 4. The boundary function 7(-) of the stopping region 
r — {{x, y) £ : v{x, y) = 0} is sketched in (d). 
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is the intersection of the straight Hnes i in (6.7) and y = ao(x). Recall from 
(11.15) that ao(-) is the boundary function of the region Aq = {{x,y) £ 
: g(x, y) < 0} = {(x, y) E : 2/ < ao(x)}. For every initial point (0o,0i) 
in M^, the sum 1 1— > x{t,4'o) + y{t,4>i), t G M+, of the coordinates of the 
parametric curve 1 1— > {x{t,(l)Q),y{t,(pi)), t G strictly decreases before 
the parametric curve meets the line i, and strictly increases thereafter; see 
Lemma 11.3 and (6.8). Finally, the point (0,(^i) with 



A(l + m) 



(11.22) 



V2{X-l) 



+ 



+ 



A(l +m) 
V2{X - 1) 



1 + 



'0' 



A(l 



m] 



-(A-1)/{A+1) 



is the initial point on the y-axis of the parametric curve t ix{t, 0),y{t, 0i)), 
t G R4. which passes through the point ((^qi^^i) (11.21). The coordinate 
4)1 in (11.22) is found by substituting the solution of x{t*,0) = (pQ for t* into 
the equation y{t*,(j)i) = (pl and solving the latter for (pi; see also Figure 6(a). 

Let us show that, under the conditions in (11.20), the "closedness" prop- 
erty in (8.4) holds. By Lemma 11.3 and (6.8), the curve Ci in Corollary 11.9 
becomes 



Ci = {{x{t,0),y{t,^i)):tGR+} 



n{{x{t,cP*o),y{t,<l)l)):tGR}; 



it is tangent to the broken line {{x,ao{x)) : x £ M+} at the point {(pQjCpi). 
Therefore, = (/iq by the same corollary, and the entrance boundary of the 
stopping region Ti = {{x,y) :vi{x,y) = 0} is dTl = {{x , ao{x)) : x e {(pQ, (A/c)V2)} 
by Corollary 11.6. Moreover, the boundary function 7i(-) of the region 
Fi = {(x,7i(x)) :7i(x) < y} is supported on [0, (A/c)\/2] and satisfies 



(11.23) 



71 (x) = aoix),x£ 



71 (x) < y{0,(pi) =(^1 



A 



V2 



and 



xG fO 



5 s^o; 



The equality follows from Corollary 11.7, and the inequality from Remark 11.10 
and the fact that the parametric curve Ci is decreasing. One can easily see 
from (11.23) and the second inequality in (11.20) that 



(11.24) 



(0, 4>d) e M+ X [cPa, 00) C S-\Ti) = {(x, S-^[^i]{x)) : x G M+}, 



7i(0). 



The restrictions of the value functions v{-,-) and vi{-,-), and therefore 



those of the boundaries dV and dVi, coincide on 



^, 00). First, observe 
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x{t, cPo) + y{t, cPi) > x{t, 0) + y{t, 4>i) > -^2 



that is, {x{t,(po),yit,(l)i)) ^ Co, 



bo,(pi) gM+ X [01, oo). 



where Co = {{x,y) G :g{x,y) < 0} is as in (4.13) and coincides with ^40. 
By the second inequahty in (11.20) and the properties of the parametric 
curves 1 1— > {x(t,(l)o),y{t,(pi)), t G ]R+ (see Section 4.2), we have 

((/>0,(/>i) G R+ X [(t)d,oo) 

r5((/.o,(/>i) GM+ X [0i,cx)) CM+ X [(/)<i,c3o) 
\(x(i,</.o),y(t,(/)i)) GM+ X [0d,oo), tGM 

Using the last two displayed equations gives that, if the initial state $o on 
a sample-path of the sufficient statistic $ = ($W,$(i)) is in M_|_ x [c/if^jOo), 
then the sample-path stays in the region x [(f)d,oo) and never returns to 
the advantageous region Co after the first jump; see Section 4.1. In fact, 



V o 5(00,01 

and therefore, 

t;(0o,0i) = -'o^'(0o,0i) 

(11.25) 



inf E, 

res 



s{4'o,<t'i) 



0, 



6o,0i) G M+ X [(f>d, oo) 



= inf / e-(^+''^^[g + f,-voS]{x{u,cj)o),y{u,cj)i))du 

iG[0,oo] Jo 

= -^0^^0(00,01) = 1'l(0O,0l), (00, 0l) G M+ X [0d,oo). 

The stopping region T = {{x,y) G M?^:v{x,y) = 0} and its boundary dF 
are determined by the value function v{-,-). Then (11.25) implies that the 
restrictions of the boundaries dV and c^Ti to the region M-|_ x [0^,00 ) also 
coincide. Therefore, the second inequality in (11.20) implies 



A 



0d<-v^<7i(O) = 7(O) and ^-^^(O) 



A* 



7(0) < 0d 



follows from (11.24). Since the boundary function S ^[7](-) of the region 
S~^(T) is decreasing [see (8.7)], the second inequality gives 

M+ X [0rf,oo)C5-i(r)={(x,y)GM^:5-i[7](x)<y}. 

But starting at any (0o, 0i) G M x [0, 0^], the parametric curves 1 1— > {x{t, 0o), 
y(t,0i)), t G M+, are increasing. Since the boundary functions 5'~"'[7](-) of 
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the regions S'~"(r) = {{x,y) G : S~^['y]{x) < y}, n G N, are also decreas- 
ing, every region S~'^{T), n G N, is "closed" in the sense of (8.4). Therefore, 
Method A after Corollary 8.4 can be used in order to calculate the value 
function v{-,-) on M^. 

Corollary 11.22. Suppose that (11.20) holds. LetBn = [/x/(/i + l)]"7i(0) 
for every 77, G N. Then the sequence M+ x [S„, oo), 77 G N, increases to x 
(0, oo), and we have v{-, •) = Vn{-, •) on x oo) for every 77 G N. 

Since {(poAi) G 1^+ x [0d,oo) C x [i?i,oo), the exit boundaries dTf 
and dT^ of the stopping regions Fi and T are the same, and 

dV- = dTf = {(x,7i(x)) :x G [0,^^)} ^ {(x, 7i(x)) : x G [0, ct>*,)}. 

From the entrance boundary dT^ = {{x,aQ{x)) :x G (0q, (A/c)\/2 )} of the 
stopping region Fi, we can obtain its exit boundary dTf and the value 
function ui(-,-) on the continuation region Ci by using Method D in Sec- 
tion 11.1; see Figures 5(a), (b) and 6(b). 

Note also that the value function 7;(-,-) =vi{-,-) is continuously differ- 
entiable on M+ x [i?i,cxD) \ 3Ff and is not differentiable on dTf by Corol- 
lary 11.5 and Lemma 11.16. Let xi = xi(7i) = min{x G M_|- : 5~^[7i](a;) = 
7i(x)} be the (smallest) intersection point of the functions S'~^[7i](-) and 
7i(-) as in (8.9). Then Corollary 8.4 implies 

{{x,-/{x)):x€ M+} n S-\Ti) = {(x, 7i(x)) : x G [0,xi]}, 

and the restriction of the boundary function 7(-) = 7i(-) to the interval 
[0,xi) is continuously differentiable by Lemma 11.15. 

Using Corollary 11.22, we can also show that the restrictions of the value 
function v{-,-) and the boundary dT of the stopping region F on the com- 
plement of the region ]R_|_ x [Bi,oo) are continuously differentiable. 

Since the sequence {i'n('i OI^gN of the value functions increases to the 
function v {■,■), all of them coincide with v{-,-) = vi(-, ■) on the region x 
[Bi,oo). On the region 1R+ x [0,Bi), they differ, but are continuously differ- 
entiable. 

In fact, since every parametric curve 1 1— > {x{t, (j)Q),y{t, (pi)), t G M_|_, start- 
ing at any point {(pOjCpi) G M4. x [0, cpd] ^ ^+ x [0, Bi] is increasing, the hy- 
pothesis Ai(r7) on page 46 holds on the region M+ x [0,(/)rf] for every G N. 

On the other hand, the third inequality in (11.20) guarantees that hy- 
pothesis A2(77) on page 46 also holds on M+ x [0, cpd] for every 77 G N. Indeed, 
every entrance boundary 9F,^^^ coincides with some part of the boundary 
dAn = {(x,a„(x)) :x G M+} of the region = {{x,y) G : [g + lu, ■ Vn o 
S]{x,y) < 0}; see Lemma 9.5. Since the sequence {an(0}nGNo of the bound- 
ary functions is increasing, the third inequality in (11.20) implies 

y < S[ao]{x) < 5K](x), n G No, (x,y) G m,4>l), (0, ^1)}. 
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Thus, by an induction on n G Nq, we can easily show that the transforma- 
tion 5((9r^_|_]^) of the entrance boundary dT'^_^_i of every stopping region 
r„_|_i is away from the exit boundary dT^_^i = dTf. Therefore, the func- 
tion {x,y) [g + /J, ■ Vn o S]{x,y) is differentiable on the entrance boundary 
9r^_l_^. The same induction, as in Section 11.2, will also prove the continuous 
differentiability of the value functions Vn{-,-), n S N and f (•, •) on the region 
M_l_ X [0,(/)f;] D M_|_ X [0,-Bi], as well as the continuous differentiability of the 
restrictions of the boundaries dTn, n G N, and dT to the set M4. x [0,1;^)^]. 

Corollary 11.23. Suppose that (11.20) holds. Then the boundary func- 
tion 7(-) of the stopping region T = {{x,y) G ■'y{x) < y} is continuously 
differentiable on its support [0,^]. The exit boundary is not empty. The 
value function v{-, ■) is continuously differentiable on \ dT^ , but not dif- 
ferentiable on dT^ . 

The interesting feature of the solutions of the problems covered under 
condition (11.20) is that the smooth- fit principle is satisfied on one connected 
proper subset of the (connected and continuously differentiable) boundary 
of the optimal stopping region, and fails on the complement of this subset. 
Moreover, the value function is continuously differentiable away from the 
boundary. 

The conditions in (11.20) are satisfied, for example, if A = 0.15, fi = 1.5 
and c = 0.7 and m = 0.9. In general, the functions 5~"'^[ao](-) and ao(") always 
intersect on the line y = x. Since 7i(-) > ao(-) and 7i(-) is decreasing, we have 
xi < (X/c) ■ (-v/2/2), with equality if and only if 



This condition is satisfied for the numbers above. As a result, we have xi = 
(A/c) • {V2/2) and 7(2;) = aoix) = x- {X/c)V2 for every x G [(/)q,xi]. The 
boundary function 7(-) is strictly above the function ao(") everywhere else. 

APPENDIX: PROOFS OF SELECTED RESULTS 

The ¥Q-infinitesimal generator A of the process $ in (4.5) Let us denote 
by A the infinitesimal generator under Pq of the process ^ = [^^^^ <|)(i)]T 
in (4.5). For every function / G C-'^'^(R+ x M4.), we have 





/(*i) = /(*o)+ E [/(*.)-/(*.-)] 



0<s<t 



(A.l) 
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and Eo<s<J/(*s) - equals 



C\ (! + -)•<& 



(1) 



Note that {Nt — /it; t > 0} is a (Po,F)-martingale. Then for every F-stopping 
time T such that 



E^»''^^|/(*,)| <oo and 



(A.2) 



/I 



•<&.1°J,(1 + -)-^ 



.(1) 



ds 



< oo, 



we have 

(A.3) Eofi^r) = f {^o) + ^o l_ Af{^s)ds, t>0, 

and Af{(po,(l)i) equals 



(A + l)</'o + 



A(l -m) 



V2 



(A.4) 



1 



A* 



(A-l)(/)i + 



A(l + m) 



/ 1-- 00, 1 + - </>! -/('/'O,'/'! 



6o,(/>i)gM+ xM+. 



Proof of Lemma 5.3. Let ti; : M be a bounded Borel function. 

Since g{-, •) > g{0, 0) = — A\/2/c in (4.12) is bounded from below, the function 
Jqw is well defined. By (5.7), 



Jw{t,(j)o,(pi) > -[-V2 + iJ,\\w\ 



V2 + 



A + /i 



for every t G [0,oo]. Since we also have Jow{(j)Q,(l)i) < Jw{0,(l)o,(pi) = 0, we 
obtain (5.9). 

Suppose now that w is also concave. For every u G ffi, the functions 
(po I— > x{u, (po) and i— > y{u, (pi) in (4.8) are linear. The mappings {(j)Q,(j)i) i— > 
S{(po,(l)i) in (5.8) and {<po,4>i) i-^ g{(po,4>i) in (4.12) are also linear. Therefore, 
the integrand in (5.7), namely {4>q, 4>i) ^ e~^'^^^^'^{g + fi-wo S){x{u, (j)o),y{u, (pi 
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is concave for every u £ [0, oo). Thus, the mappings (0o, 0i) ^ Jw{t, (</)o, (pi)), 
t e [0,00] in (5.7) are concave. Then jQw{(j)Q,4)i) = infjgp ooj Jw{t,4>o,4>i) is 
a lower envelope of concave mappings, and therefore, is a concave function 
of {4>o,(l)i) G M^. Finally, it is clear from (5.7) that wi < W2 implies that 



Proof of Corollary 5.4. The function f = has all of the prop- 
erties. The proof follows from an induction and the properties of concave 
functions. □ 



For the proof of Proposition 5.5, we shall need the following result on the 
characterization of F-stopping times; see [6], Theorem T33, page 308, and 
[9], Lemma A2.3, page 261. 

Lemma A.l. For every ¥-stopping time r and every n G Nq, there is 
an Taj^ -measurable random variable Rn : O 1-^ [0, 00] such that r A (Tn+i = 
{(Tn + Rn) A (7n+i holds fQ-a.s. on {t > an}- 



Proof of Proposition 5.5. First, we shall establish the inequality 



(A.5) Ef"'^' 



"0 



e~^*g{^t)dt>Vn{(l)o,' 







r G 5, {4>o,4>i) G 



for every n G Nq, by proving inductively on k = 1, . . . ,n + 1 that 







e-^'g{^t)dt 







(A.6) > RHSk^i 



tAs 



e-^'g{^t) dt + l{,>,}e-^'vk-i{^ 



Observe that (A.5) follows from (A.6) when we set k = n-\-\. 

If = 1, then the inequality (A.6) is satisfied as an equality, since vq = 0. 
Suppose that (A.6) holds for some 1 < < n + 1. We shall prove that it must 
also hold when k is replaced with A; + 1 . Let us denote the right-hand side 
of (A.6) by RHSk-i, and rewrite it as 



RHSk-i = RHS^^l^ + RHS'j^l^ = E^O'"^^ 



e-^'g{^t)dt 



(A.7) 



-'-{T>0-„_fc} 



tAs 



e-^'g{^t)dt 



ADAPTIVE POISSON DISORDER PROBLEM 



61 



By Lemma A.l, there is an -measurable random variable Rn-k such 

that T A Un-k+i = {(^n~k + Rn~k) A Un-k+i holds Po-almost surely on {r > 
Gn-k}- Therefore, the second expectation, denoted by RHSj^_^, in (A. 7) 
becomes 



<t>0,<Pl 



l{r>t} 



(i+fl„-fc)As 



' = °"n-fc + l 



K''^'{l{r>a^_,}e-^''-'fn^k{Rn-k,^.^.,)} 

by the strong Markov property of N , where 



/fc_i(r,</.o,</.i)^Ef°'<^^ 



"0 



rr/\ai ^ ^ 

"'0 L _ J 

= Jvk-i{r, (<^o,<Ai)) > -^o'yfe-i(0o,0i) =^^fc('^o,'Ai)- 
The (in) equalities follow from (5.3), (5.4) and (5.6), respectively. Thus 

> Eo^-*Ml{,>.„_,}e-^-"-^,(*.„_ J]. 
From (A. 6) and (A. 7), we obtain 



tA(T„ 



> RHSk^ 




\I 




Jo 






-At 








Jo 





+ RHS'^^l^ 



RHS 1^. 



This completes the proof of (A. 6) by induction on k, and (A. 5) follows by 
setting k = n + l in (A. 6). When we take the infimum of both sides in (A. 5), 
we obtain V„ > u„, n£ N. 

The reverse inequality Vn < Vn, n £ N, follows immediately from (5.11), 
since every F-stopping time S"^ is less than or equal to (T„, Pq-^.s. by con- 
struction. Therefore, we only need to establish (5.11). We shall prove it by 
induction on n G N. For n = 1, the left-hand side of (5.11) becomes 



e-^'g{^t)dt 



Juo(rg((/)o,(/>i),(/>o,(Ai 



Since Jf o(rQ((;io )</>!), </'0) ^i) < Jovo{4'o,4'i) + ^ by Remark 5.2, (5.11) holds 
for n = l. 
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Suppose that (5.11) holds for every e > for some n G N. We wiU prove 
that it also holds when n is replaced with n + 1. Since S'^j^i A cJi = rn^^(^o) A 
cji, Po-a.s., the expectation E^"'*^' [/o^"+' e"^*5(*t) di] becomes 



5f, + iA<71 



by the strong Markov property of A^, where 



nl(?>0,<?>lj 







,e/2 







< Vn{<Po,<pi)+e/2 



by the induction hypothesis. Therefore 



4'o,4>i 








(A. 







{r-„' (0o,0l)>cri| ^ -^^ 



+ e/2 



Since Jvn{rn'^{4'o,4'i),{(j)o,4'i)) < 'yn+i(9i'o, + e/2 by Remark 5.2, this 
inequality and (A. 8) prove (5.11) when n is replaced with n + 1. □ 



Proof of Proposition 5.6. Corollary 5.4 and Propositions 5.5 and 
5.1 imply that v{(j)o,4>i) = lim.„^oo 't^n(</'o, (/-i) = lim„^oo K((Ao, <Ai) = V{4>o,4>i) 
for every (cpQ, (pi) G M^. Next, let us show that V = JqV. Since (fn)n>i is a 
decreasing sequence, 

(A.9) y((/>o,(/>i) = lim ?;„((/>o,(/>i) = inf i;„((/)o, (/)i) = inf Jof„_i(<?;>o, (/>i) 

n^co n>l n>l 

for every {4>o, (pi) G R^. Since {Jvn)n>i is a decreasing sequence, and {t^njnGN 
are uniformly bounded, we have V{(po, (pi) = inf„>i jQVn-i{(po, (pi) = Jov{(po,(pi) = 
JoV{(pQ,<Pi) by the dominated convergence theorem and (A.9). Finally, since 
U < 0, we have U <Vn for every n by induction, and U < lim^^oo'^ra = ^• 
□ 
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Proof of Lemma 5.7. Let us fix a constant u>t and ((/)o,<^i) G 
Then 



Jw{u,4'o,<t>i 



(A.IO) 



9^0, 01 



tAcri 



fuAai 



On the event {fJi > t}, we have u A ai = [t + {u — t)] A [t + {ai o 0^)] = t + 
[{u — t) A (cJi o Therefore, the strong Markov property of N gives 



e: 



<t>0,<t>l - 



^0 -'-{o-i>t}e ic-o 



{u~t)AcTi 



= e-(^+^)Vu;(7x - t, <^o), y(t, </>o))) 

-E^-*^[l{,,>,|l|„>,,|e-^'^^u;(*.J]. 

The third equahty follows from the definition of Jw in (5.3) and the last 
from (4.10) and the strong Markov property. Substituting the last equation 
into (A.IO) and simplifying the rest give 

Jw{u, (Po,(l)i) = Jw{t, ((Ao, (Ai)) + e-(^+'^)* J«;(n - t, {x{t, (t>o),y{t, </.o))). 
Finally, taking the infimum of both sides over u S [t, +oo] gives (5.12). □ 

Proof of Proposition 5.11. First, let us show (5.22) for n = 1. Fix 
e > and ((/jq, G M^. By Lemma A.l, there exists a constant u G [0, oo] 
such that Ue Aai = u Aai. Then 

fuAai 



Et'''^^Mu,Aa, 



^E 



Jo 



«A(Tl 



(A.ll) 



e~^'g{^s) ds + l{„>,,}e-^'^iy(*.,: 

+ E^-^Ml{„<,^}e-^"y(*„)] 

JV{u, (</.o, cAi)) + e-(^+^)"y(x(n, ^o),y{n, <^i)) 

JuV{(j)o,(j)i), 



where the third equality follows from (5.3) and (4.10), and the fourth from 
(5.16). 
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Fix any t G [0,n). By (5.16) and (4.10) once again, we have 

JV{t, (/>o, </>i) = JtF(0o, 0i) - e-(^+^)*y (x(t, 4>o),y{t, </>!)) 

> JoV{(l,oAi)-^t'^'[lw,>t}e~^'V{^t)]- 

On the event {ai > t}, we have Us > t (otherwise, Us < t < ai would imply 
Us = u <t, contradicts with our initial choice t < u). Thus, V{^t) < 
on {(Ti > t}. Hence, 

JV{t, (t^oAi) > JoVi^o, </>!) + ee-(^+'^)" > JoV{^o, ^i) 
for every t G [0,u). Therefore, JoF((/)o, (/)i) = JuV{(po,(j)i), and (A. 11) implies 

E^°'<^MMt/,Aai] = J«^(<Ao, </>i) = ^0^(00, 0i) = V^('/'o, </'i) = <"''^MMo]. 

This completes the proof of (5.22) for n = l. Now suppose that (5.22) holds 
for some n G N, and let us show the same equality for n + 1. Note that 



Since Us A an+i = cJi + [{Us A cr„) o ^^.J on the event {Us > cxi}, the strong 
Markov property of $ at the stopping time ai will complete the proof. □ 



Proof of Proposition 5.12. Note that the sequence of random vari- 
ables 



/ e-^^g(*,)(is + e-^(^^^'^")y(*a.AaJ>-2 / e-'^'-^/lds 
Jo Jo c 



2^/2 



is bounded from below; see (4.12). By (5.22) and Fatou's lemma, we have 

y(<Ao,<Ai)>Er'^^' 



lim inf 


a 
















Jo 





for every {4>q,cI)i) G R^. The second inequality follows from (5.21). □ 
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Proof of Proposition 8.3. Let us prove (8.5) for n = 1. Take ((^o> <Ai) £ 
^"^(r). By (8.4), the curve n i— > {x{u,<j)o),y{u,4'i)), u>0, does not leave 
S~^{T). Hence, 

S{x{u,(l)o),y{u,(l)i)) eV and (F o S')(x(u, 0o), ?/(^i, 0i)) = 0, 
Then Lemma 5.6, (5.4), (5.6) and Proposition 5.5 imply that 



Since V is the limit of the decreasing sequence {KijneN; the equalities V = 
= 1^2 = • • • on (r) follow. 

On 5-i(r) n C, we have > V = Vi = ¥2 = ■ ■ ■. Therefore, S^^T) n C C 
Cfc for every fc > 1. Taking intersection of both sides with S'-"'^(r) gives 
5--'^(r) n C C 5-i(r) n for every A; > 1. To prove the opposite inclusion, 
note that F = Vfc < on ^-^(r) n Ck for every k>l. Therefore, ^-^(r) n 
Cfc ^ C, /c > 1. Intersecting both sides with the set ^-"^(r) gives S'-i(r) n 

CfcC5~i(r)nc, k>i. 

The proof of S-"(r) n T = 5-"(r) n r„ = S-"(r) n r„+i = • • • reads as 
in the previous paragraph, after every "C" is replaced by 'T," and every 
strict inequality by an equality. This completes the proof of (8.5) for n = 1. 

Suppose that (8.5) holds for some n E N, and let us prove it for n + 1. 
Take {4>q, (pi) G ^-('""^"-^^(r). Since the curve u 1— > {x{u, (f>Q),y{u, (f>i)), l^+j 
does not leave the region ^-^"^^^(r) by (8.4), we have S{x{u, (j)()),y{u, 4>i)) G 
5-"(r), ueR+, and 

{V o S){x{u, 00), y(w, 0i)) = {Vn o S){x{u, (l)o),y{u, 0i)), u £ M+, 

by the induction hypothesis. Lemma 5.6, (5.4), (5.6) and Proposition 5.5 
imply 





JoVoicpoAi) 

Vl(</>0, </>!)• 



y((Ao,<Ai) = W<^o,</'i) 





K+i(</'o,<^i)- 
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Since V is the limit of the decreasing sequence {Vn}neN, we have V = Vn+i = 
Vn+2 = ■■■ on S'"^"''~^)(r). From these equaUties follows the proof of the 
equalities of the regions in (8.5) for ?i + 1, by arguments similar to those 
presented for n = 1 above. □ 

Proof of Lemma 9.2. The obvious choices are the function a„ :M+ ^ 
]R+ and the number a„ in (9.3) and (9.4), respectively. By the discussion 
above, 

{(x,y)GM^;b + fi-VnO S]{x,y) < 0} 

= An = M.l\epi{an) 

= {{x,y)eM.l;y<an{x)} 

= {{x, y) £ [0, a„) X M+; y < a„(2;)}, 

and (9.6) follows. The proof will be complete if we show the equality in (9.5). 

Since [g + ■ Vn o S]{x,y), x G M+, is continuous, we have [g + fJ- • Vn o 
S]{x, an{x)) > for every x S M+, and the equality holds for every x £ [0, a„) 
because a„(x) >0, x £ [0,an)- Because On(-) is also continuous, the equality 
also holds for (x,y) = {an,a{an)), and 

(A.12) [g + fi- VnO S]{x,anix)) = 0, a;G[0,a„]. 

The identity (9.5) will follow immediately if we show for the 
(9.1) that 

[g + ^l■VnoS]{x,y) >0, 

(A.13) 

{x,y)£{Rl\A„)\{{x,an{x)):x£[0,an]}. 

The nonpositive function Vni-,-) is concave and equal to zero outside 
the bounded region C„. Therefore, the functions y ^ Vn{x,y), x £ M+, and 
x Vn{x,y), y £ M+, are nonpositive, concave and equal zero for every large 
real y and x, respectively. This implies that the functions y i— > Vn{x, y), x £ 
]R+, and x>-^Vn{x,y), y£M.^, are nondecreasing. Therefore, the functions 
y [g + fi ■ Vno S]{x,y), x £ M+, and x i-^ [g + fj. ■ Vn o S]{x,y), y £ R+, are 
strictly increasing since both S{x,y) and g{x,y) are strictly increasing in 
both X and y. Now (A.13) follows from (A.12). □ 

Proof of Lemma 9.7. Fix any {4>o,4'i) £ ^V^^^^j^. Then Vn+i{4'o,4>i) = 
0, and substituting ((/)q *, 0j^*) = (x(— t, (;^o)) y(— ^^i)) into (9.15) for t£ 
[0,f((/)o,(/>i)] gives 

Jtvnix{-t, (po),y{-t, (Pi)) = -e-(^+^)* Ji;„(-t, (j)o, </>!), 

(A.14) 

tE[O,r(0o, </>!)], 
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thanks to the semigroup property of •) and y(-, •). 

By the definition of the entrance boundary dV^^i in (9.10), the point 
{(fiQ, is reachable from the inside of the continuation region C„+i . Namely, 
there exists some 6 > such that t, c^o)) y(— <^i)) £ C„+i and 

fn{x{— 1^4)0), y{—t^(j)i)) = t for every t G (0,(5]. Then (9.16) implies 

> Vn+i{x{-t, ^o),y{-t, ^i)) = -e-^^+^'^'Jvni-t, (Po, 01) 

for every t€ {0,6]. Since rn{(poT'Pi) is the first time when the last function 
on the right may change its sign, we obtain 

-JVn{-t,4iO,4>l) <0, t£ {0,fni(j)0,(j)l) Ar{(j)o,(j)i)). 

Using (9.16) once again, we conclude 
Vn+i{x{-t,<j)o),y{-t,cl)i)) 

< JtVn{x{-t,(t)o),y{-t,(j3i)) 

= _e-i^+f^)ijv^(-t,cj)o,cpi)<0, t£iO,fn{cpo,'Pi)/\f{<Po,(pi))- 
Thus {{x{-t,(j)o),y{-t,cl)i));t e (0, f„((/)o, (/>i) A f((/>o, (?;>i))} C C„+i, 
rn{x{-t, 00), y(-t, 0i)) = t, Vn+i{x{-t, (j)o),y{-t, 0i)) 

= -e-(^+^)*J^;„(-t,0o,0i), 

for t £ (0, rn{4>o,4>i) A r(0O) 0i))- The third equation follows from the second 
and (A. 14), and the second from the first and the fact {x{t, x{—t, (l)o)),y{t, y{—t, 
4>i))) = i4'Oi4'i) £ r. Taking the limit in the third equation as t increases to 
'f'n{4>o,4>i) gives 

run+i(x(-t,0o),y(-t,0i))|t=r;„(0o,0i) =0 and 1 

I {x{-rni4>o,4>i),4>o),y{-rn{4>o, 4>i),4>i)) e ^r^+i J 

if f„(0o,0i) <r(0o,0i)- 
Finally, every (cpo, (pi) £ C„+iU9r^_,_i is reachable from (cpo, cpi) = r„(0O) 01 ) S 
9r^_,_i on the entrance boundary by {{x{t,4>o),y{t,(j)i));t G [0, r„,(0O) 0i)]} 
which is contained (possibly, excluding end-points) in the continuation re- 
gion Cn+l. □ 

Proof of Corollary 11.6. By Lemma 11.4, the function (0o,0i) ^ 
ro(0o5<^i) is continuous on the continuation region (0O)0i) S Ci. Therefore, 
the entrance boundary dTf is the image of the continuous mapping [see the 
definition in (9.10)] 

(00, 0i) ^ (2;('^o(</'o,0i),0i),7i(y('^o('/'o,0i),0i))), (00, 0l) G Ci, 

from the connected Ci into Mi. Thus the set dT^ is a connected subset of 
Ml. 
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Since the parametric curves 1 1— > (/iq), 0)), 4>o £ M+ starting on the 
X-axis are increasing, the points on the boundary dTi where these curves 
meet the boundary belong to the entrance boundary dTi ; see also Figure 1 . 
Hence {(a;, 71 (a;)) : x E ^ dTf for some < 5 < .^1. Then the connect- 

edness of dTl gives (11.14) with = inf{x G M+ : (a;,7i(x)) G dTl}. 

Indeed, the point (Ci5 7i(Ci)) does not belong to the entrance boundary 
arf. Suppose it does. Then {(x(-t,^f),y(-t,7i(^f)));t G (0,<5]} C Ci for 
some (5 > 0. Let ((/>o,(/>i) G Ci be the point in the middle of the vertical 
line-segment connecting the points (5, ^f),y(— (5, 7i(^f))) and 5, .^f), 
7i(x(— (5,^f))). Then we have x{6,(j)o) = x(— (5, ^f)) = and y{d,(pi) > 
y{S,y{-^,7i{Ci))) =li{Ci) =7i{x{S,<))o)) since the mapping (j)^ y{t,4>) is in- 
creasing for every t G M. Therefore, (j)Q),y{5, 0i)) G Fi and < rQ{(j)Q,(pi) < 
5. Thus we have (2;(ro(0o, '/'i), <^o), yiroi4'o,4>i),4>i)) G "^rf , but x{ro{<j)o,4>i), 
(po) < (/>o) = [the mapping (/> 1— > </)) is increasing for every t G M]. 
This contradicts the minimality of . □ 

Proof of Corollary 11.12. Suppose > and fix any (po G [0,^f). 
Let ^0 ^ (l/2)(0o + ef)- Then {^oHiiM) e dTf, and ^i{4>o) > 7i(0o) > 
71 (^f) since 7i(-) is strictly decreasing on its support. Then the set B = 
[0,(^0) ^ (7i(<^o)!Oo) is an open neighborhood of (<^Oi7i(<^o)) such that for 
every (0Oi 0i) G .B n Ci, we have 

<£< ro(<^o,<^i) <r <oo, 

where r = inf{i > : y(t, 7i((^o)) <7i(Ci)} and f = mf{t > 0:x{t,0) >^i}. 
This completes the proof of the first part. 

Now let {(j)Q,(j)i) G dTl be a point on the entrance boundary. Take any 

convergent sequence {((/>q"\ (/)j^"^)}„gN in the continuation region Ci whose 
limit is the boundary point {(f)Q, (pi). Since ro(-, ■) <f (see above) on Ci, the 

sequence {ro{(j)Q^\(j)i^^)}neN is bounded and has a convergent subsequence. 
We shall conclude the proof of the second part by showing that every con- 
vergent subsequence of the sequence {ro((/)o"'\ (/>j^"^)}ngN has the same limit 
0. 

Without changing the notation, suppose that {?^o('/'o"^' '^i"^)}"eN con- 
verges to some finite number rg > 0. Since the functions {(j)o,(j)i) ^ vi{(f)Q,(f)i) 
and (t, (/)o, (/)i) I— > JvQ{t,4)Q,4)i) are continuous, we have 

= z;i(,^o,<Ai)= lim z;i(0S"\ </.(")) = lim J^;o(ro(4"\ ./-S"^), 0^"^ <A^^) 

n— *-oo ^ ^ n— >oo ^ ^ x / u _l / 

= Jvo{r^, (po,(pi)= e-(^+^)*Go(t, (/.q, c^i) dt. 
Jo 

If we show that Go{t,(pQ,(j)i) > for every t > 0, then rg = follows. 

However, t = is a point of increase for the function 1 1— > Go(t, i?5>o, 0i). 
Since {<j)o,(pi) G dTl = {{x,ao{x)) :x G ('?i,'^i)} by Corollary 11.7, and the 
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boundary function ao(-) of the region Aq = {{x,y) £ ■.[g + ^-vooS]{x,y) < 
0} is strictly decreasing, there exists some 6 > such that {x{t, (po),y{t, (pi)) G 
^0 ^ Ci for every t S [—5,0). Therefore, 

G{t,(j)o,(l)i) = [g + fi-VQO S]{x{t,cl)o),y{t,(j)i)) < = Go{0,cl)o,(j)i), 

te[-6,0). 

Then Lemma 11.3 imphes that GQ{t,(j)Q,(j)i) > for every t> and completes 
the proof of tq = 0. □ 

Proof of Lemma 11.13. If = 0, then c\{dTl) = {{x,ji{x)):x e 
[0,^1]} = dVi by Corollary 11.6. In the remainder, suppose that > and 
fix any (po £ [0,^i). The boundary point ((/lo, 7i((/'o)) is not included in the 
entrance boundary OF^ We shall prove that it is an exit boundary point; 
namely, there exists some 6 > such that [see (9.10)] 

(A.15) (x(t,(/.o),y(t,7i(</'o)))GCi ytei0,6]. 

Since the boundary 7i(-) is strictly decreasing on its support [0,^i], we 
have 



0<^o<ei =^ 7i(0o)> 71(^1)- 
Then there is always a sequence of points {{(l)^\(f>^^'')}nt^f^ ^ Ci such that 
0q"^ = and 0^"^ > 71(^1) for every n G N, and 
lim = I 7^(0o)- 

n—*oo 

Namely, the sequence {((/)q"\ (/)^"^)}„gN "increases" to the point {4>o,7i{4'o)) 
along the vertical line passing through the point ((/>o, 7i((^o))- For every n G 
N, we have 

and 

(x(ro(4"\4"^'Ai"^y(ro(<^f,"\<Ai"^4"^)) e 9F^ 

By Corollary 11.12, the sequence {ro(i;^o"'\ i;^>["^)}„gN is bounded. Therefore, 
it has a convergent subsequence; we shall denote it by the same notation 
and its limit by tq. The functions Jvo{-,-,-), x{-,-), y{-,-) and fi(-,-) are 
continuous, and vi{(j)o,ji{(po)) = 0. Therefore, taking limits of the displayed 
equations above gives 

(A.16) = Jt;o(ro,(/'o,7i(0o)) and {x{ro,<f)o),y{ro,1i{(l)o))) ^ cl{dTl). 
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The second expression implies that a;(ro,(/>o) > Cf- We shah prove that the 
inequahty is strict, and therefore, 

(A.17) (x(ro,</.o),y(ro,7i(0o)))ean- 

Let us assume that x(ro, </>o) = ■^f • Then the second expression in (A. 16) im- 
phes that (x(ro,(/)o),y(ro,7i((/'o))) = (?i,7i(Cf))- Thus ((/'o, 7i('/'o)) is on the 
curve Ci given by (11.16). Then Corollary 11.9 implies that G{t, (poj^iicpo)) > 
for every t^r^. Since tq > 0, this implies that Juo(^0; ^^O) 7i(<^o)) = 
j^" e~^^'^^^^GQ{s,(i)Q,'^i{(l)Q)) ds is strictly positive. But this contradicts the 
first equality in (A. 16). Therefore, we must have 2;(ro,i;^>o) > Cf, and (A.17) 
is correct. 

Now we are ready to prove (A. 15). Since < ^f, we have [g + ■ vq o 
S"] ((/>o , 7i ((^0 ) ) > by Corollary 11.11. Because the mapping [g + ^■vqoS]{-,-) 
is continuous, there exists some ro > (5 > such that 

G'o(i,'/'o,7i('/'o)) = [g + ^-t■voo S]{x{t,(l3o),y{t,-ii{(l)Q)))>{), te [0,6]. 

Then for every t £ {0,6] we have 

t;o(3;(t,(/)o),y(i,7i(0o))) 

< Jvo{ro -t,x{t,cl)o),y{t,-fi{(po))) 

ro—t 

e-{A+/>[^ + . t-o o S]{xiu, x{t, cPo)),y{u, y{t, 71 ((/-o)))) du 





pro 

^(A+;.)i / e-(A+M)«[^ + ^.^Qo5](x(t,<Ao),y(i,7i('^o)))d« 



Jvo(.ro,(l)o,li{M)- / e-(^+^)"Go(n,0o,7i('/'o))c?n 







<0. 



=0 

Therefore, (A.15) holds and (0o, 7i(<?^'o)) e ^Ff • □ 

Proof of Lemma 11.14. There is nothing to prove if = 0. Therefore, 
suppose > 0. Let Bi be the union of the continuation region Ci and the 
open subset of [0,i^f) x strictly below the curve Ci in Corollary 11.9. 
Then Bi is open and Ci U dVi C Bi; see Remark 11.10. Define 



I ro(<^o, 0i) = mf{t > : (x(t, </.o), y(t, </>i)) G 9rf } I 

\vi{(j)0,(pl) = Jvo{fo{(j)o,(j)i),(j)o,(j)i) J 

for every {(1)0, 4>i) G -Bi- 

Then 

ro{(l)o,(l)i) =fo{4>o,4'i) and ?;i(</>o, (/)i) = vi(</'o, </>i), 

(A.18) 

(0o,</'i)GCiU9rf. 
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Let us show that fo(-,-), and therefore, vi{-,-) are continuously differen- 
tiable on Bi. The infimum fo{4>o,4'i) is finite and strictly positive for every 
((/>o,<Ai) By (9.12), Go{fo{(j)o,(j)i),(po,(pi) equals 

[g + fi-VQO S]{x{ro{(j)o,(j)i),(l)o),y{ro{(l)o,(l)i),(l)i)) = 0, 

(A.19) 

{(po,(l)i)eBi. 

The mapping (t, (/jq, i;^>i) ^ Go(t, (/'o, is continuously diff'erentiable. If 
(A.20) AG'o(i,</>o,</'i)|t=fo(<^oA)^0' {4>o,(l)i)eBi, 

then Theorem 11.2 implies that, in an open neighborhood in Bi of every 
{(po^cpi), the equation Go(t, (/'o, 0i) =0 determines t = t{(f)Q,(f)i) implicitly 
as a function of ((/)o, </>!), and this function is continuously differentiable. 
In every neighborhood, these solutions must then coincide with fo((/)05 <^i)- 
Therefore, ro{(j>o,4'i) is continuously differentiable on Bi. Then the function 
vi{4>o, (pi) is continuously differentiable on Bi since Jvq{-, •, •) is continuously 
differentiable on ]Ri]_. 

Now fix any {(j)Q,cj)i) £ Bi and assume DtGo{fo{(j)o, (pi) , (j)o, = 0. Then 
the function 1 1-^ Go{t, (pQ,(pi) has a local minimum at t = fo(</>o, (Ai)- Lemma 11.3 
and (A.19) imply that Go(t, (/)0) </>!) > for every t / fo((/>o, (/^i). Therefore, 
the parametric curve 

{(x(i, </.o), y(t, (pi)) : t G M} n C \ ^0 

does not intersect but touches the boundary BAq. Then this curve has to 
be the same as Ci in Corollary 11.16, and {(pQ,(pi) G Ci. But this contradicts 
(0O)0i) ^ Bi, since Remark 11.10 and the description of Bi show that Ci H 
Bi = 0. Therefore, (A.20) holds. 

Now let us show that 7i(-) is continuously differentiable on [0,^f). Fix any 
(Pq G [0,ef). Then (0o,7i(</'o)) e dT^ C Bi and ^i(0o,7i(</'o)) = by (A.18). 
The function vi{-,-) is continuously differentiable on Bi. Therefore, the re- 
sult will again follow from the implicit function theorem (Theorem 11.2) if 
we show that i^0^{;i((/)o,7i((/)o)) 7^0. However, L'0^{;i(i;^O)7i(</'o)) equals 

DsVi{(pQ,-ii{(po))=D,vi{(t)Q,-fi{(pQ)) = lim D7vi{(pQ,(pi) 

= lim D^^vi{(pQ,(pi) 

\ _ g-(A«+i)n)(</'o,<^i) 

= hm > 0. 

Mii{4>o) /i+1 

The second equality follows from (A.18), and the third from the concav- 
ity of vi{-,-). The fourth and the fifth follow from Corollary 11.5. Finally, 
the limit at the end is strictly positive since 7'o(-,-) is bounded away from 
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zero in the intersection of Ci with some neighborhood of (0O)7i(</'o)) by 
Corollary 11.12. □ 

Proof of Lemma 11.15. The result follows from Corollary 11.7 if 
= 0. Therefore, suppose > 0. Then the boundary function 7i(-) is con- 
tinuously differentiable on [0, ) U {^f, ^i) by Corollary 11.7 and Lemma 11.14. 
We need to show that x i— > 71 (x) is continuously differentiable at x = . 

Recall that the function 7i(-) is convex. Therefore, the left derivative 
D~ji{-) and the right derivative D^7i(-) of the function 7i(-) exist and are 
left- and right-continuous, respectively, at x = ^f. Thus 

hm 1)71 (x) = hm Z)-7i(x) = D"7i(^?) 

(A.21) 

The continuity of the derivative D^yi^-) of the function 7i(-) at x = will 
follow immediately from the existence of the derivative of 7i(-) at x = . 

Now recall from Corollary 11.9 and Remark 11.10 that the point (^f , 71(^1)) 
is on the parametric curve Ci, which lies above {(x,7i(x)):x € M+} and 
touches it at the point (Cf) 7i (■?!))• Therefore, for every t>0 and s > 

y(0,7i(^f))-?/H,7i(^f)) ^ 7i(^(0,g!))-7i(^(-^,ef)) 
x(0,ef)-x(-i,e!) - x(0,ef)-x(-t,ef) 

x(s,ef)-x(o,ef) 

^ j/(g,7i(Cf))-|/(0,7i(^f)) 

- x(s,ef)-x(o,ef) 

When we take the limit as 1 1 and s | 0, we obtain 

Aj/(o,7i(e!)) . .,e^ . n+o, (^-\ < Dtyio,iim) 

Note that the terms on the far left and far right are the same. Therefore, 
Z)~7i(^f ) = D~^^i{(^f) and the derivative of the boundary function 7i(-) at 
X = ,^1 exists. □ 

Proof of Lemma 11.16. Since is concave, the left derivatives 

D^o^ii-r), -C^i^^i(t) and the right derivatives D^^vi{-,-), D^^vi{-,-) exist 
and are left- and right-continuous on the boundary dT, respectively. Because 
vi{-, •) vanishes on Fi, and the function 7i(-) is strictly decreasing, we have 

Dd;oM4>o,4>i) > Df vi{4>o,4>i) = 0, 

(A.22) 
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(A.23) 



(<Ao,</'i)G9ri\{(0,7i(0))}. 
(0o,</'i)G5ri\{(ei,O)}. 



For every boundary point (0o,</'i) G 5ri\{(0,7i(0))} and any {((/>^ ')}n&n 
C Ci such that lim„_»oo 4^^^^ =T </'o and 0^"^ = 4'i foi' every n G N, we have 

(A.24) = Jim 



lim 



l-exp{-(/.-l)ro(4"\4"))} 



The second and the tliird equahties follow from Corollary 11.5. The function 
ro(-,-) is continuous on the entrance boundary dV\ and is bounded away 
from zero in some neighborhood of every point on the exit boundary 5rf ; 
see Corollary 11.12. Therefore, the limit on the right in (A.24) equals zero 
for every point {cj)Q, (pi) on the entrance boundary dTl and is strictly positive 
for every point ((/>o,(/>i) on the exit boundary dTf. 

Thus, for every ((/)o, </>i) G dT\, the equality in (A. 22), and as a result of a 
similar argument, the equality in (A.23) are attained. Therefore, the partial 
derivatives -D0gfi(-,-) and D^^vi[-^-) exist at every ((/)o,(/>i) G dV\ and are 
continuous since D^qVi{-,-) = D^^vi{-,-) is both left- and right-continuous 
near the entrance boundary 9rf . 

However, if {4>q, tpi) is a point on the exit boundary dTf, then the inequal- 
ities in (A.23) and (A.24) are strict. Namely, the fi(-, •) is not differentiable 
on the exit boundary dTf. □ 



Acknowledgment We are grateful to the referee for detailed comments 
that helped us improve the manuscript. 



REFERENCES 

[1] Bayraktar, E. and Dayanik, S. (2006). Poisson disorder problem with exponential 
penalty for delay. Math. Oper. Res. 31 217-233. 

[2] Bayraktar, E., Dayanik, S. and Karatzas, I. (2004). Adaptive Pois- 
son disorder problem. Working paper, Princeton Univ. Available at 
http: / / www.princeton.edu / ~sdayanik / papers /bayes.pdf . 

[3] Bayraktar, E., Dayanik, S. and Karatzas, I. (2005). The standard Poisson dis- 
order problem revisited. Stochastic Process. Appl. 115 1437-1450. MR2158013 

[4] Beibel, M. (1997). Sequential change-point detection in continuous time when the 
post-change drift is unknown. Bernoulli 3 457-478. MR1483699 



74 



E. BAYRAKTAR, S. DAYANIK AND I. KARATZAS 



[5] Beibel, M. and Lerche, H. R. (2003). Sequential Bayes detection of trend changes. 

In Foundations of Statistical Inference (Y. Haitovsky, H. R. Lerche and Y. Ritov, 
eds.) 117-130. Physica, Heidelberg. MR2017819 

[6] Bremaud, p. (1981). Point Processes and Queues. Springer, New York. MR8260058 

[7] Clarke, F. H., Ledyaev, Y. S., Stern, R. J. and Wolenski, P. R. (1998). Non- 
smooth Analysis and Control Theory. Springer, New York. MR9949001 

[8] Davis, M. H. A. (1976). A note on the Poisson disorder problem. Banach Center 
Publ. 1 65-72. 

[9] Davis, M. H. A. (1993). Markov Models and Optimization. Chapman and HaU, 
London. MR1283589 

[10] Galchuk, L. L and RozovSKll, B. L. (1971). The disorder problem for a Poisson 
process. Theory Probab. Appl. 16 729-734. MR0297028 

[11] Gugerli, U. S. (1986). Optimal stopping of a piecewise-deterministic Markov pro- 
cess. Stochastics 19 221-236. MR0872462 

[12] Krantz, S. G. and Parks, H. R. (2002). The Implicit Function Theorem. 
Birkhauser, Boston. MR0326001 

[13] Liptser, R. S. and Shiryaev, A. N. (2001). Statistics of Random Processes I. 
Springer, Berlin. MR0160001 

[14] 0KSENDAL, B. (1998). Stochastic Differential Equations. Springer, Berlin. 
MR9960119 

[15] Peskir, G. and Shiryaev, A. N. (2002). Solving the Poisson disorder problem. In 
Advances in Finance and Stochastics (K. Sandmann and P. J. Schonbucher, 
eds.) 295-312. Springer, Berlin. MR1929384 

[16] Protter, M. H. and Morrey, Jr., C. B. (1991). A First Course in Real Analysis, 
2nd ed. Springer, New York. MR1123269 

[17] ROCKAFELLAR, R. T. (1997). Convcx Analysis. Princeton Univ. Press. MR1451876 

E. BAYRAKTAR S. DAYANIK 

Department of Mathematics Department of Operations Research 

University of Michigan and Financial Engineering 

Ann Arbor, Michigan 48109 and the Bendheim Center for Finance 

USA Princeton University 

E-MAIL: erhan@umich.cdu Princeton, New Jersey 08544 

USA 

E-MAIL: sdayanik@princeton.edu 

I. Karatzas 

Departments of Mathematics 

AND Statistics 
Columbia University 
New York, New York 10027 
USA 

E-MAIL: ik@math.columbia.edu 



